Max level shown:
1 [ Aug 26 21:45:41 Enabled. ]
2 [ Aug 26 21:45:41 Rereading configuration. ]
3 [ Aug 26 21:45:46 Rereading configuration. ]
4 [ Aug 26 21:45:58 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/omicron-nexus/bin/nexus /var/svc/manifest/site/nexus/config.toml &"). ]
5 [ Aug 26 21:45:58 Method "start" exited with status 0. ]
6 note: configured to log to "/dev/stdout"
72025-08-26T21:45:58.850ZDEBGnexus: registered DTrace probes
82025-08-26T21:45:58.854ZINFOnexus: setting up nexus server file = nexus/src/lib.rs:79
92025-08-26T21:45:58.867ZINFOnexus (ServerContext): registering Oso class class = Action file = nexus/auth/src/authz/oso_generic.rs:68
102025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = AuditLog file = nexus/auth/src/authz/oso_generic.rs:68
112025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = AnyActor file = nexus/auth/src/authz/oso_generic.rs:68
122025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = AuthenticatedActor file = nexus/auth/src/authz/oso_generic.rs:68
132025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = BlueprintConfig file = nexus/auth/src/authz/oso_generic.rs:68
142025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = Database file = nexus/auth/src/authz/oso_generic.rs:68
152025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = DnsConfig file = nexus/auth/src/authz/oso_generic.rs:68
162025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = Fleet file = nexus/auth/src/authz/oso_generic.rs:68
172025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = Inventory file = nexus/auth/src/authz/oso_generic.rs:68
182025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = IpPoolList file = nexus/auth/src/authz/oso_generic.rs:68
192025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = ConsoleSessionList file = nexus/auth/src/authz/oso_generic.rs:68
202025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = DeviceAuthRequestList file = nexus/auth/src/authz/oso_generic.rs:68
212025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = QuiesceState file = nexus/auth/src/authz/oso_generic.rs:68
222025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = SiloCertificateList file = nexus/auth/src/authz/oso_generic.rs:68
232025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = SiloIdentityProviderList file = nexus/auth/src/authz/oso_generic.rs:68
242025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = SiloUserList file = nexus/auth/src/authz/oso_generic.rs:68
252025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = SiloUserSessionList file = nexus/auth/src/authz/oso_generic.rs:68
262025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = SiloUserTokenList file = nexus/auth/src/authz/oso_generic.rs:68
272025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = UpdateTrustRootList file = nexus/auth/src/authz/oso_generic.rs:68
282025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = TargetReleaseConfig file = nexus/auth/src/authz/oso_generic.rs:68
292025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = AlertClassList file = nexus/auth/src/authz/oso_generic.rs:68
302025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = Project file = nexus/auth/src/authz/oso_generic.rs:68
312025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = Disk file = nexus/auth/src/authz/oso_generic.rs:68
322025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = Snapshot file = nexus/auth/src/authz/oso_generic.rs:68
332025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = ProjectImage file = nexus/auth/src/authz/oso_generic.rs:68
342025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = AffinityGroup file = nexus/auth/src/authz/oso_generic.rs:68
352025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = AntiAffinityGroup file = nexus/auth/src/authz/oso_generic.rs:68
362025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = Instance file = nexus/auth/src/authz/oso_generic.rs:68
372025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = IpPool file = nexus/auth/src/authz/oso_generic.rs:68
382025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = InstanceNetworkInterface file = nexus/auth/src/authz/oso_generic.rs:68
392025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = Vpc file = nexus/auth/src/authz/oso_generic.rs:68
402025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = VpcRouter file = nexus/auth/src/authz/oso_generic.rs:68
412025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = InternetGateway file = nexus/auth/src/authz/oso_generic.rs:68
422025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = InternetGatewayIpPool file = nexus/auth/src/authz/oso_generic.rs:68
432025-08-26T21:45:58.868ZINFOnexus (ServerContext): registering Oso class class = InternetGatewayIpAddress file = nexus/auth/src/authz/oso_generic.rs:68
442025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = RouterRoute file = nexus/auth/src/authz/oso_generic.rs:68
452025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = VpcSubnet file = nexus/auth/src/authz/oso_generic.rs:68
462025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = FloatingIp file = nexus/auth/src/authz/oso_generic.rs:68
472025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = Image file = nexus/auth/src/authz/oso_generic.rs:68
482025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = SiloImage file = nexus/auth/src/authz/oso_generic.rs:68
492025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = AddressLot file = nexus/auth/src/authz/oso_generic.rs:68
502025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = Blueprint file = nexus/auth/src/authz/oso_generic.rs:68
512025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = LoopbackAddress file = nexus/auth/src/authz/oso_generic.rs:68
522025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = Certificate file = nexus/auth/src/authz/oso_generic.rs:68
532025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = ConsoleSession file = nexus/auth/src/authz/oso_generic.rs:68
542025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = DeviceAuthRequest file = nexus/auth/src/authz/oso_generic.rs:68
552025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = DeviceAccessToken file = nexus/auth/src/authz/oso_generic.rs:68
562025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = PhysicalDisk file = nexus/auth/src/authz/oso_generic.rs:68
572025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = Rack file = nexus/auth/src/authz/oso_generic.rs:68
582025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = SshKey file = nexus/auth/src/authz/oso_generic.rs:68
592025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = Silo file = nexus/auth/src/authz/oso_generic.rs:68
602025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = SiloUser file = nexus/auth/src/authz/oso_generic.rs:68
612025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = SiloGroup file = nexus/auth/src/authz/oso_generic.rs:68
622025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = SupportBundle file = nexus/auth/src/authz/oso_generic.rs:68
632025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = IdentityProvider file = nexus/auth/src/authz/oso_generic.rs:68
642025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = SamlIdentityProvider file = nexus/auth/src/authz/oso_generic.rs:68
652025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = Sled file = nexus/auth/src/authz/oso_generic.rs:68
662025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = TufRepo file = nexus/auth/src/authz/oso_generic.rs:68
672025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = TufArtifact file = nexus/auth/src/authz/oso_generic.rs:68
682025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = TufTrustRoot file = nexus/auth/src/authz/oso_generic.rs:68
692025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = Alert file = nexus/auth/src/authz/oso_generic.rs:68
702025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = AlertReceiver file = nexus/auth/src/authz/oso_generic.rs:68
712025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = WebhookSecret file = nexus/auth/src/authz/oso_generic.rs:68
722025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = Zpool file = nexus/auth/src/authz/oso_generic.rs:68
732025-08-26T21:45:58.869ZINFOnexus (ServerContext): registering Oso class class = Service file = nexus/auth/src/authz/oso_generic.rs:68
742025-08-26T21:45:58.870ZINFOnexus (ServerContext): registering Oso class class = UserBuiltin file = nexus/auth/src/authz/oso_generic.rs:68
752025-08-26T21:45:58.870ZINFOnexus (ServerContext): full Oso configuration config = #\n# Oso configuration for Omicron\n# This file is augmented by generated snippets.\n#\n\n#\n# ACTOR TYPES AND BASIC RULES\n#\n\n# `AnyActor` includes both authenticated and unauthenticated users.\nactor AnyActor {}\n\n# An `AuthenticatedActor` has an identity in the system. All of our operations\n# today require that an actor be authenticated.\nactor AuthenticatedActor {}\n\n# For any resource, `actor` can perform action `action` on it if they're\n# authenticated and their role(s) give them the corresponding permission on that\n# resource.\nallow(actor: AnyActor, action: Action, resource) if\n actor.authenticated and\n has_permission(actor.authn_actor.unwrap(), action.to_perm(), resource);\n\n# Define role relationships\nhas_role(actor: AuthenticatedActor, role: String, resource: Resource)\n\tif resource.has_role(actor, role);\n\n#\n# ROLES AND PERMISSIONS IN THE FLEET/SILO/PROJECT HIERARCHY\n#\n# We define the following permissions for most resources in the system:\n#\n# - "create_child": required to create child resources (of any type)\n#\n# - "list_children": required to list child resources (of all types) of a\n# resource\n#\n# - "modify": required to modify or delete a resource\n#\n# - "read": required to read a resource\n#\n# We define the following predefined roles for only a few high-level resources:\n# the Fleet (see below), Silo, Organization, and Project. The specific roles\n# are oriented around intended use-cases:\n#\n# - "admin": has all permissions on the resource\n#\n# - "collaborator": has "read", "list_children", and "create_child", plus\n# the "admin" role for child resources. The idea is that if you're an\n# Organization Collaborator, you have full control over the Projects within\n# the Organization, but you cannot modify or delete the Organization itself.\n#\n# - "viewer": has "read" and "list_children" on a resource\n#\n# Below the Project level, permissions are granted via roles at the Project\n# level. For example, for someone to be able to create, modify, or delete any\n# Instances, they must be granted project.collaborator, which means they can\n# create, modify, or delete _all_ resources in the Project.\n#\n# The complete set of predefined roles:\n#\n# - fleet.admin (superuser for the whole system)\n# - fleet.collaborator (can manage Silos)\n# - fleet.viewer (can read most non-siloed resources in the system)\n# - silo.admin (superuser for the silo)\n# - silo.collaborator (can create and own Organizations)\n# - silo.viewer (can read most resources within the Silo)\n# - organization.admin (complete control over an organization)\n# - organization.collaborator (can manage Projects)\n# - organization.viewer (can read most resources within the Organization)\n# - project.admin (complete control over a Project)\n# - project.collaborator (can manage all resources within the Project)\n# - project.viewer (can read most resources within the Project)\n#\n# Outside the Silo/Organization/Project hierarchy, we (currently) treat most\n# resources as nested under Fleet or else a synthetic resource (see below). We\n# do not yet support role assignments on anything other than Fleet, Silo,\n# Organization, or Project.\n#\n\n# "Fleet" is a global singleton representing the whole system. The name comes\n# from the idea described in RFD 24, but it's not quite right. This probably\n# should be more like "Region" or "AvailabilityZone". The precise boundaries\n# have not yet been figured out.\nresource Fleet {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\troles = [\n\t # Roles that can be attached by users\n\t "admin",\n\t "collaborator",\n\t "viewer",\n\n\t # Internal-only roles\n\t "external-authenticator"\n\t];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n}\n\n# For fleets specifically, roles can be conferred by roles on the user's Silo.\nhas_role(actor: AuthenticatedActor, role: String, _: Fleet) if\n\tsilo_role in actor.confers_fleet_role(role) and\n\thas_role(actor, silo_role, actor.silo.unwrap());\n\nresource Silo {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Permissions implied by roles on this resource's parent (Fleet). Fleet\n\t# privileges allow a user to see and potentially administer the Silo,\n\t# but they do not give anyone permission to look at anything inside the\n\t# Silo. To achieve this, we use permission rules here. (If we granted\n\t# Fleet administrators _roles_ on the Silo, then those would cascade\n\t# into the Silo as well.)\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "collaborator" on "parent_fleet";\n\n\t# external authenticator has to create silo users\n\t"list_children" if "external-authenticator" on "parent_fleet";\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\n\nhas_relation(fleet: Fleet, "parent_fleet", silo: Silo)\n\tif silo.fleet = fleet;\n\n# As a special case, all authenticated users can read their own Silo. That's\n# not quite the same as having the "viewer" role. For example, they cannot list\n# Organizations in the Silo.\n#\n# One reason this is necessary is because if an unprivileged user tries to\n# create an Organization using "POST /organizations", they should get back a 403\n# (which implies they're able to see /organizations, which is essentially seeing\n# the Silo itself) rather than a 404. This behavior isn't a hard constraint\n# (i.e., you could reasonably get a 404 for an API you're not allowed to call).\n# Nor is the implementation (i.e., we could special-case this endpoint somehow).\n# But granting this permission is the simplest way to keep this endpoint's\n# behavior consistent with the rest of the API.\n#\n# This rule is also used to determine if a user can list the identity providers\n# in the Silo (which they should be able to), since that's predicated on being\n# able to read the Silo.\n#\n# It's unclear what else would break if users couldn't see their own Silo.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif silo in actor.silo;\n\nresource Project {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Roles implied by roles on this resource's parent (Silo)\n\trelations = { parent_silo: Silo };\n\t"admin" if "collaborator" on "parent_silo";\n\t"viewer" if "viewer" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", project: Project)\n\tif project.silo = silo;\n\n#\n# GENERAL RESOURCES OUTSIDE THE SILO/PROJECT HIERARCHY\n#\n# Many resources use snippets of Polar generated by the `authz_resource!` Rust\n# macro. Some resources require custom Polar code. Those appear here.\n#\n\nresource Certificate {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Fleet-level and silo-level roles both grant privileges on certificates.\n\t"read" if "admin" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", certificate: Certificate)\n\tif certificate.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", certificate: Certificate)\n\tif certificate.silo.fleet = fleet;\n\nresource SiloUser {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\t# Fleet and Silo administrators can manage a Silo's users. This is one\n\t# of the only areas of Silo configuration that Fleet Administrators have\n\t# permissions on.\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "read" on "parent_fleet";\n\t"read" if "read" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", user: SiloUser)\n\tif user.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", user: SiloUser)\n\tif user.silo.fleet = fleet;\n\n# authenticated actors have all permissions on themselves\nhas_permission(actor: AuthenticatedActor, _perm: String, silo_user: SiloUser)\n if actor.equals_silo_user(silo_user);\n\nhas_permission(actor: AuthenticatedActor, "read", silo_user: SiloUser)\n if silo_user.silo in actor.silo;\n\nresource SiloGroup {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\trelations = { parent_silo: Silo };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", group: SiloGroup)\n\tif group.silo = silo;\n\nresource SshKey {\n\tpermissions = [ "read", "modify" ];\n\trelations = { silo_user: SiloUser };\n\n\t"read" if "read" on "silo_user";\n\t"modify" if "modify" on "silo_user";\n}\nhas_relation(user: SiloUser, "silo_user", ssh_key: SshKey)\n\tif ssh_key.silo_user = user;\n\nresource IdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", identity_provider: IdentityProvider)\n\tif identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: IdentityProvider)\n\tif collection.silo.fleet = fleet;\n\nresource SamlIdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", saml_identity_provider: SamlIdentityProvider)\n\tif saml_identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SamlIdentityProvider)\n\tif collection.silo.fleet = fleet;\n\n#\n# SYNTHETIC RESOURCES OUTSIDE THE SILO HIERARCHY\n#\n# The resources here do not correspond to anything that appears explicitly in\n# the API or is stored in the database. These are used either at the top level\n# of the API path (e.g., "/v1/system/ip-pools") or as an implementation detail of the system\n# (in the case of console sessions and "Database"). The policies are\n# either statically-defined in this file or driven by role assignments on the\n# Fleet. None of these resources defines their own roles.\n#\n\n# Describes the quiesce state of a particular Nexus instance.\n#\n# These authz checks must not require the database. We grant this directly to\n# callers of the internal API.\nresource QuiesceState {\n\tpermissions = [ "read", "modify" ];\n}\nhas_permission(USER_INTERNAL_API: AuthenticatedActor, "read", _q: QuiesceState);\nhas_permission(\n USER_INTERNAL_API: AuthenticatedActor,\n "modify",\n _q: QuiesceState\n);\n\n# Describes the policy for reading and modifying DNS configuration\n# (both internal and external)\nresource DnsConfig {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t# "external-authenticator" requires these permissions because that's the\n\t# context that Nexus uses when creating and deleting Silos. These\n\t# operations necessarily need to read and modify DNS configuration.\n\t"read" if "external-authenticator" on "parent_fleet";\n\t"modify" if "external-authenticator" on "parent_fleet";\n\t# "admin" on the parent fleet also gets these permissions, primarily for\n\t# the test suite.\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", dns_config: DnsConfig)\n\tif dns_config.fleet = fleet;\n\n# Describes the policy for accessing blueprints\nresource BlueprintConfig {\n\tpermissions = [\n\t "list_children", # list blueprints\n\t "create_child", # create blueprint\n\t "read", # read the current target\n\t "modify", # change the current target\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"read" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", list: BlueprintConfig)\n\tif list.fleet = fleet;\n\n# Describes the policy for accessing "/v1/system/update/trust-roots" in the API\nresource UpdateTrustRootList {\n\tpermissions = [ "list_children", "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"list_children" if "viewer" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: UpdateTrustRootList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for accessing blueprints\nresource TargetReleaseConfig {\n\tpermissions = [\n\t "read", # read the current target release\n\t "modify", # change the current target release\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", resource: TargetReleaseConfig)\n\tif resource.fleet = fleet;\n\n# Describes the policy for reading and modifying low-level inventory\nresource Inventory {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", inventory: Inventory)\n\tif inventory.fleet = fleet;\n\n# Describes the policy for accessing "/v1/system/ip-pools" in the API\nresource IpPoolList {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "create_child",\n\t];\n\n\t# Fleet Administrators can create or modify the IP Pools list.\n\trelations = { parent_fleet: Fleet };\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n\n\t# Fleet Viewers can list IP Pools\n\t"list_children" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", ip_pool_list: IpPoolList)\n\tif ip_pool_list.fleet = fleet;\n\n# Any authenticated user can create a child of a provided IP Pool.\n# This is necessary to use the pools when provisioning instances.\nhas_permission(actor: AuthenticatedActor, "create_child", ip_pool: IpPool)\n\tif silo in actor.silo and silo.fleet = ip_pool.fleet;\n\n# Describes the policy for reading and writing the audit log \nresource AuditLog {\n\tpermissions = [\n\t "list_children", # retrieve audit log\n\t "create_child", # create audit log entry\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\n\t# Fleet viewers can read the audit log\n\t"list_children" if "viewer" on "parent_fleet";\n}\n\n# Any actor should be able to write to the audit log because we need to be able\n# to write to the log from any request, authenticated or not. Audit log writes\n# are always a byproduct of other operations: there are no endpoints that allow\n# the user to write to the log deliberately. Note we use AuthenticatedActor\n# because we don't really mean unauthenticated -- in the case of login\n# operations, we use the external authenticator actor that creates the session\n# to authorize the audit log write.\nhas_permission(_actor: AuthenticatedActor, "create_child", _audit_log: AuditLog);\n\nhas_relation(fleet: Fleet, "parent_fleet", audit_log: AuditLog)\n\tif audit_log.fleet = fleet;\n\n# Describes the policy for creating and managing web console sessions.\nresource ConsoleSessionList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: ConsoleSessionList)\n\tif collection.fleet = fleet;\n\n# Allow silo admins to delete and list user sessions\nresource SiloUserSessionList {\n permissions = [ "modify", "list_children" ];\n relations = { parent_silo: Silo };\n\n # A silo admin can modify (e.g., delete) a user's sessions.\n "modify" if "admin" on "parent_silo";\n\n # A silo admin can list a user's sessions.\n "list_children" if "admin" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", authn_list: SiloUserSessionList)\n if authn_list.silo_user.silo = silo;\n\n# give users 'modify' and 'list_children' on their own sessions\nhas_permission(actor: AuthenticatedActor, "modify", authn_list: SiloUserSessionList)\n if actor.equals_silo_user(authn_list.silo_user);\nhas_permission(actor: AuthenticatedActor, "list_children", authn_list: SiloUserSessionList)\n if actor.equals_silo_user(authn_list.silo_user);\n\n# Allow silo admins to delete and list user access tokens\nresource SiloUserTokenList {\n permissions = [ "modify", "list_children" ];\n relations = { parent_silo: Silo };\n\n # A silo admin can modify (e.g., delete) a user's tokens.\n "modify" if "admin" on "parent_silo";\n\n # A silo admin can list a user's tokens.\n "list_children" if "admin" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", authn_list: SiloUserTokenList)\n if authn_list.silo_user.silo = silo;\n\n# give users 'modify' and 'list_children' on their own tokens\nhas_permission(actor: AuthenticatedActor, "modify", authn_list: SiloUserTokenList)\n if actor.equals_silo_user(authn_list.silo_user);\nhas_permission(actor: AuthenticatedActor, "list_children", authn_list: SiloUserTokenList)\n if actor.equals_silo_user(authn_list.silo_user);\n\n# Describes the policy for creating and managing device authorization requests.\nresource DeviceAuthRequestList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: DeviceAuthRequestList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing Silo certificates\nresource SiloCertificateList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Both Fleet and Silo administrators can see and modify the Silo's\n\t# certificates.\n\t"list_children" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloCertificateList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloCertificateList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo identity providers\nresource SiloIdentityProviderList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the identity providers in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's identity provider\n\t# configuration. This is one of the only areas of Silo configuration\n\t# that Fleet Administrators have permissions on. This is also one of\n\t# the only cases where we need to look two levels up the hierarchy to\n\t# see if somebody has the right permission. For most other things,\n\t# permissions cascade down the hierarchy so we only need to look at the\n\t# parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloIdentityProviderList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloIdentityProviderList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo users (mostly intended for\n# API-managed users)\nresource SiloUserList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the users in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's users. This is\n\t# one of the only areas of Silo configuration that Fleet Administrators\n\t# have permissions on. This is also one of the few cases (so far) where\n\t# we need to look two levels up the hierarchy to see if somebody has the\n\t# right permission. For most other things, permissions cascade down the\n\t# hierarchy so we only need to look at the parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloUserList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloUserList)\n\tif collection.silo.fleet = fleet;\n\n# These rules grants the external authenticator role the permissions it needs to\n# read silo users and modify their sessions. This is necessary for login to\n# work.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif has_role(actor, "external-authenticator", silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\n\n# All authenticated users can read and delete device authn requests because\n# by necessity these operations happen before we've figured out what user (or\n# even Silo) the device auth is associated with. Any user can claim a device\n# auth request with the right user code (that's how it works) -- it's the user\n# code and associated logic that prevents unauthorized access here.\nhas_permission(_actor: AuthenticatedActor, "read", _device_auth: DeviceAuthRequest);\nhas_permission(_actor: AuthenticatedActor, "modify", _device_auth: DeviceAuthRequest);\n\nhas_permission(actor: AuthenticatedActor, "read", device_token: DeviceAccessToken)\n\tif has_role(actor, "external-authenticator", device_token.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", identity_provider: IdentityProvider)\n\tif has_role(actor, "external-authenticator", identity_provider.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", saml_identity_provider: SamlIdentityProvider)\n\tif has_role(actor, "external-authenticator", saml_identity_provider.silo.fleet);\n\n# Describes the policy for who can access the internal database.\nresource Database {\n\tpermissions = [\n\t # "query" is required to perform any query against the database,\n\t # whether a read or write query. This is checked when an operation\n\t # checks out a database connection from the connection pool.\n\t #\n\t # Any authenticated user gets this permission. There's generally\n\t # some other authz check involved in the database query. For\n\t # example, if you're querying the database to "read" a "Project", we\n\t # should also be checking that. So why do we do this at all? It's\n\t # a belt-and-suspenders measure so that if we somehow introduced an\n\t # unauthenticated code path that hits the database, it cannot be\n\t # used to DoS the database because we won't allow the operation to\n\t # make the query. (As long as the code path _is_ authenticated, we\n\t # can use throttling mechanisms to prevent DoS.)\n\t "query",\n\n\t # "modify" is required to populate database data that's delivered\n\t # with the system. It should also be required for schema changes,\n\t # when we support those. This is separate from "query" so that we\n\t # cannot accidentally invoke these code paths from API calls and\n\t # other general functions.\n\t "modify"\n\t];\n}\n\n# All authenticated users have the "query" permission on the database.\nhas_permission(_actor: AuthenticatedActor, "query", _resource: Database);\n\n# The "db-init" user is the only one with the "modify" permission.\nhas_permission(USER_DB_INIT: AuthenticatedActor, "modify", _resource: Database);\nhas_permission(USER_DB_INIT: AuthenticatedActor, "create_child", _resource: IpPoolList);\n# It also has "admin" on the internal silo to populate it with built-in resources.\n# TODO-completeness: actually limit to just internal silo and not all silos\nhas_role(USER_DB_INIT: AuthenticatedActor, "admin", _silo: Silo);\n\n# Allow the internal API admin permissions on all silos.\nhas_role(USER_INTERNAL_API: AuthenticatedActor, "admin", _silo: Silo);\n\nresource WebhookSecret {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_alert_receiver: AlertReceiver };\n\n\t"read" if "read" on "parent_alert_receiver";\n\t"modify" if "modify" on "parent_alert_receiver";\n}\n\nhas_relation(rx: AlertReceiver, "parent_alert_receiver", secret: WebhookSecret)\n\tif secret.alert_receiver = rx;\n\nresource AlertClassList {\n\tpermissions = [ "list_children" ];\n\trelations = { parent_fleet: Fleet };\n\n\t"list_children" if "viewer" on "parent_fleet";\n}\n\nhas_relation(fleet: Fleet, "parent_fleet", collection: AlertClassList)\n\tif collection.fleet = fleet;\n\n\n\n resource Disk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Disk)\n if child.project = parent;\n \n\n resource Snapshot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Snapshot)\n if child.project = parent;\n \n\n resource ProjectImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: ProjectImage)\n if child.project = parent;\n \n\n resource AffinityGroup {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: AffinityGroup)\n if child.project = parent;\n \n\n resource AntiAffinityGroup {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: AntiAffinityGroup)\n if child.project = parent;\n \n\n resource Instance {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Instance)\n if child.project = parent;\n \n\n resource IpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: IpPool)\n if child.fleet = fleet;\n \n\n resource InstanceNetworkInterface {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Instance\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InstanceNetworkInterface)\n if has_relation(project, "containing_project", child.instance);\n\n has_relation(parent: Instance, "parent", child: InstanceNetworkInterface)\n if child.instance = parent;\n \n\n resource Vpc {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Vpc)\n if child.project = parent;\n \n\n resource VpcRouter {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcRouter)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcRouter)\n if child.vpc = parent;\n \n\n resource InternetGateway {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGateway)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: InternetGateway)\n if child.vpc = parent;\n \n\n resource InternetGatewayIpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpPool)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpPool)\n if child.internet_gateway = parent;\n \n\n resource InternetGatewayIpAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpAddress)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpAddress)\n if child.internet_gateway = parent;\n \n\n resource RouterRoute {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: VpcRouter\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: RouterRoute)\n if has_relation(project, "containing_project", child.vpc_router);\n\n has_relation(parent: VpcRouter, "parent", child: RouterRoute)\n if child.vpc_router = parent;\n \n\n resource VpcSubnet {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcSubnet)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcSubnet)\n if child.vpc = parent;\n \n\n resource FloatingIp {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: FloatingIp)\n if child.project = parent;\n \n\n resource Image {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: Image)\n if child.silo = parent;\n \n\n resource SiloImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: SiloImage)\n if child.silo = parent;\n \n\n resource AddressLot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: AddressLot)\n if child.fleet = fleet;\n \n\n resource Blueprint {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Blueprint)\n if child.fleet = fleet;\n \n\n resource LoopbackAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: LoopbackAddress)\n if child.fleet = fleet;\n \n\n\n resource ConsoleSession {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: ConsoleSession)\n if child.fleet = fleet;\n \n\n resource DeviceAuthRequest {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAuthRequest)\n if child.fleet = fleet;\n \n\n resource DeviceAccessToken {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAccessToken)\n if child.fleet = fleet;\n \n\n resource PhysicalDisk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: PhysicalDisk)\n if child.fleet = fleet;\n \n\n resource Rack {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Rack)\n if child.fleet = fleet;\n \n\n\n\n\n\n resource SupportBundle {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: SupportBundle)\n if child.fleet = fleet;\n \n\n\n\n resource Sled {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Sled)\n if child.fleet = fleet;\n \n\n resource TufRepo {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufRepo)\n if child.fleet = fleet;\n \n\n resource TufArtifact {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufArtifact)\n if child.fleet = fleet;\n \n\n resource TufTrustRoot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufTrustRoot)\n if child.fleet = fleet;\n \n\n resource Alert {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Alert)\n if child.fleet = fleet;\n \n\n resource AlertReceiver {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: AlertReceiver)\n if child.fleet = fleet;\n \n\n\n resource Zpool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Zpool)\n if child.fleet = fleet;\n \n\n resource Service {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Service)\n if child.fleet = fleet;\n \n\n resource UserBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: UserBuiltin)\n if child.fleet = fleet;\n file = nexus/auth/src/authz/oso_generic.rs:89
762025-08-26T21:45:58.889ZINFOnexus (ServerContext): Setting up resolver using DNS servers for subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344::, width: 48 } } file = nexus/src/context.rs:223
772025-08-26T21:45:58.893ZINFOnexus (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:144
782025-08-26T21:45:58.895ZINFOnexus (ServerContext): Setting up qorb database pool from DNS dns_addrs = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = nexus/src/context.rs:279
792025-08-26T21:45:58.909ZDEBGnexus (ServerContext): registered USDT probes
802025-08-26T21:45:59.119ZINFOnexus: Database schema version is up to date desired_version = 183.0.0 file = nexus/db-queries/src/db/datastore/db_metadata.rs:143 found_version = 183.0.0
812025-08-26T21:45:59.123ZINFOnexus (SEC): SEC running file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/steno-0.4.1/src/sec.rs:813 sec_id = 4996dcc4-c530-4f64-b407-fa372462bd9e
822025-08-26T21:45:59.185ZDEBGnexus (clickhouse-client): registered USDT probes id = bd809b3c-059c-4abc-afb6-167347d6f53e
832025-08-26T21:45:59.202ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
842025-08-26T21:45:59.233ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
852025-08-26T21:45:59.234ZDEBGnexus (DataLoader): attempting to create built-in users
862025-08-26T21:45:59.235ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
872025-08-26T21:45:59.235ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
882025-08-26T21:45:59.238ZINFOnexus (MgsUpdateDriver): starting MgsUpdateDriver file = nexus/mgs-updates/src/driver.rs:146
892025-08-26T21:45:59.280ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /bgtasks versions = all
902025-08-26T21:45:59.280ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /bgtasks/activate versions = all
912025-08-26T21:45:59.280ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /bgtasks/view/{bgtask_name} versions = all
922025-08-26T21:45:59.280ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /clickhouse/policy versions = all
932025-08-26T21:45:59.280ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /clickhouse/policy versions = all
942025-08-26T21:45:59.281ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stop-request versions = all
952025-08-26T21:45:59.281ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stopped versions = all
962025-08-26T21:45:59.281ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair/{repair_id}/progress versions = all
972025-08-26T21:45:59.281ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-finish versions = all
982025-08-26T21:45:59.281ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-start versions = all
992025-08-26T21:45:59.281ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /demo-saga versions = all
1002025-08-26T21:45:59.281ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /demo-saga/{demo_saga_id}/complete versions = all
1012025-08-26T21:45:59.281ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /deployment/blueprints/all versions = all
1022025-08-26T21:45:59.281ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = DELETE path = /deployment/blueprints/all/{blueprint_id} versions = all
1032025-08-26T21:45:59.281ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /deployment/blueprints/all/{blueprint_id} versions = all
1042025-08-26T21:45:59.281ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /deployment/blueprints/import versions = all
1052025-08-26T21:45:59.281ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /deployment/blueprints/regenerate versions = all
1062025-08-26T21:45:59.281ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /deployment/blueprints/target versions = all
1072025-08-26T21:45:59.281ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /deployment/blueprints/target versions = all
1082025-08-26T21:45:59.281ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = PUT path = /deployment/blueprints/target/enabled versions = all
1092025-08-26T21:45:59.281ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /deployment/chicken-switches versions = all
1102025-08-26T21:45:59.281ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /deployment/chicken-switches versions = all
1112025-08-26T21:45:59.281ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /deployment/chicken-switches/{version} versions = all
1122025-08-26T21:45:59.281ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /deployment/update-status versions = all
1132025-08-26T21:45:59.281ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /disk/{disk_id}/remove-read-only-parent versions = all
1142025-08-26T21:45:59.281ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = PUT path = /disks/{disk_id} versions = all
1152025-08-26T21:45:59.281ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /experimental/v1/system/support-bundles versions = all
1162025-08-26T21:45:59.281ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /experimental/v1/system/support-bundles versions = all
1172025-08-26T21:45:59.281ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = DELETE path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
1182025-08-26T21:45:59.281ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
1192025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = PUT path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
1202025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
1212025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
1222025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
1232025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
1242025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/index versions = all
1252025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /instances/{instance_id}/migrate versions = all
1262025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /metrics/collectors versions = all
1272025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /metrics/collectors/{collector_id}/producers versions = all
1282025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /metrics/producers versions = all
1292025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /mgs-updates versions = all
1302025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /nat/ipv4/changeset/{from_gen} versions = all
1312025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /oximeter/read-policy versions = all
1322025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /oximeter/read-policy versions = all
1332025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /physical-disk/expunge versions = all
1342025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /probes/{sled} versions = all
1352025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /quiesce versions = all
1362025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /quiesce versions = all
1372025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = PUT path = /racks/{rack_id}/initialization-complete versions = all
1382025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /sagas versions = all
1392025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /sagas/{saga_id} versions = all
1402025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /sled-agents/{sled_id} versions = all
1412025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /sled-agents/{sled_id} versions = all
1422025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /sled-agents/{sled_id}/firewall-rules-update versions = all
1432025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /sleds/add versions = all
1442025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /sleds/expunge versions = all
1452025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /sleds/uninitialized versions = all
1462025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = PUT path = /switch/{switch_id} versions = all
1472025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = GET path = /v1/ping versions = all
1482025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = PUT path = /vmms/{propolis_id} versions = all
1492025-08-26T21:45:59.282ZDEBGnexus (dropshot_internal): registered endpoint local_addr = [fd00:1122:3344:101::a]:12221 method = POST path = /volume/{volume_id}/remove-read-only-parent versions = all
1502025-08-26T21:45:59.283ZINFOnexus (dropshot_internal): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:328 local_addr = [fd00:1122:3344:101::a]:12221
1512025-08-26T21:45:59.283ZDEBGnexus (dropshot_internal): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::a]:12221
1522025-08-26T21:45:59.283ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
1532025-08-26T21:45:59.283ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
1542025-08-26T21:45:59.289ZWARNnexus: Cannot look up rack: Object (of type ById(e37a46f2-f789-4ca4-9841-f8c6dd705964)) not found: rack file = nexus/src/app/rack.rs:781
1552025-08-26T21:45:59.357ZINFOnexus (DataLoader): created 0 built-in users file = nexus/db-queries/src/db/datastore/silo_user.rs:393
1562025-08-26T21:45:59.357ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1572025-08-26T21:45:59.359ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1582025-08-26T21:45:59.359ZDEBGnexus (DataLoader): attempting to create built-in role assignments
1592025-08-26T21:45:59.359ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1602025-08-26T21:45:59.359ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1612025-08-26T21:46:00.010ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:46597
1622025-08-26T21:46:00.031ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::d]:46597 req_id = f4a3960f-1dbe-4d4b-95a6-c9d60404bcdc roles = RoleSet { roles: {} } uri = /metrics/collectors
1632025-08-26T21:46:00.032ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::d]:46597 req_id = f4a3960f-1dbe-4d4b-95a6-c9d60404bcdc resource = Database result = Ok(()) uri = /metrics/collectors
1642025-08-26T21:46:00.050ZINFOnexus (DataLoader): created 0 built-in role assignments file = nexus/db-queries/src/db/datastore/role.rs:55
1652025-08-26T21:46:00.050ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1662025-08-26T21:46:00.052ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1672025-08-26T21:46:00.052ZDEBGnexus (DataLoader): attempting to create built-in silos
1682025-08-26T21:46:00.052ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1692025-08-26T21:46:00.052ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1702025-08-26T21:46:00.181ZINFOnexus (DataLoader): created 0 built-in silos file = nexus/db-queries/src/db/datastore/silo.rs:101
1712025-08-26T21:46:00.181ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1722025-08-26T21:46:00.182ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1732025-08-26T21:46:00.202ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1742025-08-26T21:46:00.204ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1752025-08-26T21:46:00.204ZDEBGnexus (DataLoader): attempting to create built-in projects
1762025-08-26T21:46:00.204ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1772025-08-26T21:46:00.204ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1782025-08-26T21:46:00.235ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1792025-08-26T21:46:00.236ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1802025-08-26T21:46:00.252ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1812025-08-26T21:46:00.253ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1822025-08-26T21:46:00.340ZINFOnexus: registered new oximeter metric collection server address = [fd00:1122:3344:101::d]:12223 collector_id = f843ff0e-eef8-49c4-b428-23e15df1def3 file = nexus/src/app/oximeter.rs:41
1832025-08-26T21:46:00.342ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 327307 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::d]:46597 req_id = f4a3960f-1dbe-4d4b-95a6-c9d60404bcdc response_code = 204 uri = /metrics/collectors
1842025-08-26T21:46:01.212ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1852025-08-26T21:46:01.214ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
1862025-08-26T21:46:01.214ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1872025-08-26T21:46:01.214ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1882025-08-26T21:46:01.255ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1892025-08-26T21:46:01.255ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1902025-08-26T21:46:01.257ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1912025-08-26T21:46:01.259ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
1922025-08-26T21:46:01.259ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1932025-08-26T21:46:01.259ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1942025-08-26T21:46:01.291ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
1952025-08-26T21:46:01.291ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
1962025-08-26T21:46:01.293ZWARNnexus: Cannot look up rack: Object (of type ById(e37a46f2-f789-4ca4-9841-f8c6dd705964)) not found: rack file = nexus/src/app/rack.rs:781
1972025-08-26T21:46:01.367ZINFOnexus (DataLoader): created built-in services project file = nexus/db-queries/src/db/datastore/project.rs:122
1982025-08-26T21:46:01.367ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
1992025-08-26T21:46:01.368ZDEBGnexus (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2002025-08-26T21:46:01.368ZDEBGnexus (DataLoader): attempting to create built-in VPCs
2012025-08-26T21:46:01.368ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2022025-08-26T21:46:01.368ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2032025-08-26T21:46:01.386ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2042025-08-26T21:46:01.386ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2052025-08-26T21:46:01.389ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2062025-08-26T21:46:01.389ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2072025-08-26T21:46:01.436ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2082025-08-26T21:46:01.436ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2092025-08-26T21:46:01.437ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2102025-08-26T21:46:01.438ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2112025-08-26T21:46:01.439ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2122025-08-26T21:46:01.442ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2132025-08-26T21:46:01.442ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2142025-08-26T21:46:01.442ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2152025-08-26T21:46:01.491ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2162025-08-26T21:46:01.491ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2172025-08-26T21:46:01.493ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2182025-08-26T21:46:01.494ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2192025-08-26T21:46:01.495ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2202025-08-26T21:46:01.498ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2212025-08-26T21:46:01.498ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2222025-08-26T21:46:01.499ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2232025-08-26T21:46:01.558ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2242025-08-26T21:46:01.558ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2252025-08-26T21:46:01.561ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2262025-08-26T21:46:01.561ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2272025-08-26T21:46:01.601ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2282025-08-26T21:46:01.602ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2292025-08-26T21:46:01.603ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2302025-08-26T21:46:01.606ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
2312025-08-26T21:46:01.606ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2322025-08-26T21:46:01.607ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2332025-08-26T21:46:01.610ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2342025-08-26T21:46:01.610ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2352025-08-26T21:46:01.613ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2362025-08-26T21:46:01.613ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2372025-08-26T21:46:01.614ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2382025-08-26T21:46:01.615ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2392025-08-26T21:46:01.618ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2402025-08-26T21:46:01.618ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2412025-08-26T21:46:01.619ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2422025-08-26T21:46:01.623ZDEBGnexus (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
2432025-08-26T21:46:01.623ZDEBGnexus (DataLoader): roles roles = RoleSet { roles: {} }
2442025-08-26T21:46:01.623ZDEBGnexus (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2452025-08-26T21:46:01.660ZERROnexus (DataLoader): gave up trying to populate built-in PopulateBuiltinVpcs error_message = InternalError { internal_message: "Unknown diesel error creating VpcRouter called \\"system\\": Record not found" } file = nexus/src/populate.rs:126
2462025-08-26T21:46:01.660ZERROnexus: populate failed file = nexus/src/app/mod.rs:551
2472025-08-26T21:46:01.660ZINFOnexus: activating background tasks file = nexus/src/app/mod.rs:557
2482025-08-26T21:46:01.663ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 1 reason = Timeout
2492025-08-26T21:46:01.663ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 1 reason = Timeout
2502025-08-26T21:46:01.663ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 1 reason = Timeout
2512025-08-26T21:46:01.663ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 1 reason = Timeout
2522025-08-26T21:46:01.663ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 1 reason = Timeout
2532025-08-26T21:46:01.663ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 1 reason = Timeout
2542025-08-26T21:46:01.663ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
2552025-08-26T21:46:01.663ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 1 reason = Timeout
2562025-08-26T21:46:01.663ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 1 reason = Timeout
2572025-08-26T21:46:01.663ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
2582025-08-26T21:46:01.663ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2592025-08-26T21:46:01.663ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 1 reason = Timeout
2602025-08-26T21:46:01.663ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
2612025-08-26T21:46:01.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
2622025-08-26T21:46:01.663ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 1 reason = Timeout
2632025-08-26T21:46:01.663ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 1 reason = Timeout
2642025-08-26T21:46:01.663ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
2652025-08-26T21:46:01.663ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:61 reason = no config nor servers
2662025-08-26T21:46:01.663ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:61 reason = no config nor servers
2672025-08-26T21:46:01.663ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:36:01.663846827 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
2682025-08-26T21:46:01.664ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 1 reason = Timeout
2692025-08-26T21:46:01.664ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:82 reason = no blueprint
2702025-08-26T21:46:01.664ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 54.871\u{b5}s iteration = 1 status = {"error":"no config nor servers"}
2712025-08-26T21:46:01.664ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 45.971\u{b5}s iteration = 1 status = {"error":"no config nor servers"}
2722025-08-26T21:46:01.665ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 1 reason = Timeout
2732025-08-26T21:46:01.665ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 1 reason = Timeout
2742025-08-26T21:46:01.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2752025-08-26T21:46:01.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2762025-08-26T21:46:01.666ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
2772025-08-26T21:46:01.671ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 7.478328ms iteration = 1 status = {"error":"no blueprint"}
2782025-08-26T21:46:01.671ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 1 reason = Timeout
2792025-08-26T21:46:01.672ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
2802025-08-26T21:46:01.672ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
2812025-08-26T21:46:01.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
2822025-08-26T21:46:01.672ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 165.794\u{b5}s iteration = 1 status = "Disabled"
2832025-08-26T21:46:01.673ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 1 reason = Timeout
2842025-08-26T21:46:01.673ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 1 reason = Timeout
2852025-08-26T21:46:01.673ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 1 reason = Timeout
2862025-08-26T21:46:01.673ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2872025-08-26T21:46:01.673ZWARNnexus (BackgroundTasks): Blueprint rendezvous: skipped background_task = blueprint_rendezvous file = nexus/src/app/background/tasks/blueprint_rendezvous.rs:52 reason = no blueprint
2882025-08-26T21:46:01.673ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 92.022\u{b5}s iteration = 1 status = {"error":"no blueprint"}
2892025-08-26T21:46:01.673ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 1 reason = Timeout
2902025-08-26T21:46:01.673ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 1 reason = Timeout
2912025-08-26T21:46:01.673ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = crdb_node_id_collector file = nexus/src/app/background/tasks/crdb_node_id_collector.rs:78 reason = no blueprint
2922025-08-26T21:46:01.673ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 4.88\u{b5}s iteration = 1 status = {"error":"no blueprint"}
2932025-08-26T21:46:01.673ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 1 reason = Timeout
2942025-08-26T21:46:01.673ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
2952025-08-26T21:46:01.674ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
2962025-08-26T21:46:01.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2972025-08-26T21:46:01.677ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 1 reason = Timeout
2982025-08-26T21:46:01.677ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 1 reason = Timeout
2992025-08-26T21:46:01.677ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3002025-08-26T21:46:01.678ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
3012025-08-26T21:46:01.678ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 1 reason = Timeout
3022025-08-26T21:46:01.678ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
3032025-08-26T21:46:01.678ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3042025-08-26T21:46:01.681ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 1 reason = Timeout
3052025-08-26T21:46:01.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
3062025-08-26T21:46:01.684ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3072025-08-26T21:46:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
3082025-08-26T21:46:01.685ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 1 reason = Timeout
3092025-08-26T21:46:01.685ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 1 reason = Timeout
3102025-08-26T21:46:01.685ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
3112025-08-26T21:46:01.685ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 1 reason = Timeout
3122025-08-26T21:46:01.685ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 1 reason = Timeout
3132025-08-26T21:46:01.685ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 1 reason = Timeout
3142025-08-26T21:46:01.685ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
3152025-08-26T21:46:01.685ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
3162025-08-26T21:46:01.685ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3172025-08-26T21:46:01.685ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3182025-08-26T21:46:01.685ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
3192025-08-26T21:46:01.685ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
3202025-08-26T21:46:01.685ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 865, tv_nsec: 544441506 } })
3212025-08-26T21:46:01.685ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 865, tv_nsec: 545072330 } })
3222025-08-26T21:46:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
3232025-08-26T21:46:01.686ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 1 reason = Timeout
3242025-08-26T21:46:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
3252025-08-26T21:46:01.686ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 1 reason = Timeout
3262025-08-26T21:46:01.686ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 1 reason = Timeout
3272025-08-26T21:46:01.686ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 1 reason = Timeout
3282025-08-26T21:46:01.686ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
3292025-08-26T21:46:01.686ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
3302025-08-26T21:46:01.686ZDEBGnexus (BackgroundTasks): activating background_task = saga_recovery iteration = 1 reason = Timeout
3312025-08-26T21:46:01.686ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 1 reason = Timeout
3322025-08-26T21:46:01.686ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
3332025-08-26T21:46:01.686ZINFOnexus (SagaQuiesceHandle): saga recovery pass starting file = nexus/types/src/quiesce.rs:319
3342025-08-26T21:46:01.686ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 1 reason = Timeout
3352025-08-26T21:46:01.686ZDEBGnexus (SagaRecoverer): listing candidate sagas for recovery
3362025-08-26T21:46:01.686ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
3372025-08-26T21:46:01.686ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 1 reason = Timeout
3382025-08-26T21:46:01.686ZDEBGnexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
3392025-08-26T21:46:01.686ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
3402025-08-26T21:46:01.686ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 1 reason = Timeout
3412025-08-26T21:46:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
3422025-08-26T21:46:01.686ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 1 reason = Timeout
3432025-08-26T21:46:01.686ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
3442025-08-26T21:46:01.686ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 1 reason = Timeout
3452025-08-26T21:46:01.686ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
3462025-08-26T21:46:01.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
3472025-08-26T21:46:01.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3482025-08-26T21:46:01.693ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 1 reason = Timeout
3492025-08-26T21:46:01.694ZINFOnexus (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal dns_group = internal file = nexus/src/app/background/tasks/dns_servers.rs:113
3502025-08-26T21:46:01.694ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 2 reason = Dependency
3512025-08-26T21:46:01.694ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 30.914297ms iteration = 1 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
3522025-08-26T21:46:01.694ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
3532025-08-26T21:46:01.694ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 20.86\u{b5}s iteration = 2 status = {"error":"no config"}
3542025-08-26T21:46:01.694ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3552025-08-26T21:46:01.694ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
3562025-08-26T21:46:01.694ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 8.564464ms iteration = 1 status = {"disabled":true,"errors":[],"sps":[]}
3572025-08-26T21:46:01.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
3582025-08-26T21:46:01.694ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
3592025-08-26T21:46:01.695ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3602025-08-26T21:46:01.695ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 1 reason = Timeout
3612025-08-26T21:46:01.695ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
3622025-08-26T21:46:01.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3632025-08-26T21:46:01.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
3642025-08-26T21:46:01.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3652025-08-26T21:46:01.697ZINFOnexus (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external dns_group = external file = nexus/src/app/background/tasks/dns_servers.rs:113
3662025-08-26T21:46:01.697ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 33.802112ms iteration = 1 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
3672025-08-26T21:46:01.697ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 2 reason = Dependency
3682025-08-26T21:46:01.697ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
3692025-08-26T21:46:01.697ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 6.76\u{b5}s iteration = 2 status = {"error":"no config"}
3702025-08-26T21:46:01.699ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3712025-08-26T21:46:01.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
3722025-08-26T21:46:01.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
3732025-08-26T21:46:01.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
3742025-08-26T21:46:01.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
3752025-08-26T21:46:01.703ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3762025-08-26T21:46:01.703ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 865, tv_nsec: 562884702 } })
3772025-08-26T21:46:01.703ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
3782025-08-26T21:46:01.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
3792025-08-26T21:46:01.710ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 865, tv_nsec: 569152783 } })
3802025-08-26T21:46:01.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
3812025-08-26T21:46:01.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
3822025-08-26T21:46:01.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
3832025-08-26T21:46:01.711ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
3842025-08-26T21:46:01.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3852025-08-26T21:46:01.711ZDEBGnexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
3862025-08-26T21:46:01.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
3872025-08-26T21:46:01.714ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3882025-08-26T21:46:01.714ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3892025-08-26T21:46:01.714ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3902025-08-26T21:46:01.714ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
3912025-08-26T21:46:01.714ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
3922025-08-26T21:46:01.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3932025-08-26T21:46:01.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
3942025-08-26T21:46:01.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
3952025-08-26T21:46:01.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
3962025-08-26T21:46:01.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3972025-08-26T21:46:01.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3982025-08-26T21:46:01.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3992025-08-26T21:46:01.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
4002025-08-26T21:46:01.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
4012025-08-26T21:46:01.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
4022025-08-26T21:46:01.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
4032025-08-26T21:46:01.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
4042025-08-26T21:46:01.716ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 53.162968ms iteration = 1 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
4052025-08-26T21:46:01.718ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
4062025-08-26T21:46:01.718ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
4072025-08-26T21:46:01.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
4082025-08-26T21:46:01.719ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4092025-08-26T21:46:01.721ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4102025-08-26T21:46:01.722ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
4112025-08-26T21:46:01.722ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
4122025-08-26T21:46:01.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
4132025-08-26T21:46:01.723ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4142025-08-26T21:46:01.726ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4152025-08-26T21:46:01.726ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4162025-08-26T21:46:01.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4172025-08-26T21:46:01.726ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
4182025-08-26T21:46:01.726ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4192025-08-26T21:46:01.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4202025-08-26T21:46:01.734ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4212025-08-26T21:46:01.737ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
4222025-08-26T21:46:01.737ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
4232025-08-26T21:46:01.738ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
4242025-08-26T21:46:01.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
4252025-08-26T21:46:01.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
4262025-08-26T21:46:01.747ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4272025-08-26T21:46:01.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4282025-08-26T21:46:01.750ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4292025-08-26T21:46:01.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4302025-08-26T21:46:01.773ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
4312025-08-26T21:46:01.773ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 87.197396ms iteration = 1 status = {"by_rx":{},"error":null}
4322025-08-26T21:46:01.780ZINFOnexus (BackgroundTasks): Physical Disk Adoption: Rack not yet initialized background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:81 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
4332025-08-26T21:46:01.781ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 107.612566ms iteration = 1 status = {"error":"rack not yet initialized: e37a46f2-f789-4ca4-9841-f8c6dd705964"}
4342025-08-26T21:46:01.783ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
4352025-08-26T21:46:01.783ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4362025-08-26T21:46:01.784ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d566ed46-fae2-4b5e-8f34-d448f387d4b5", "content-length": "26", "date": "Tue, 26 Aug 2025 21:46:01 GMT"} })
4372025-08-26T21:46:01.784ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:36:01.663846827 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
4382025-08-26T21:46:01.784ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 121.496819ms iteration = 1 status = {"expiration":"2025-08-26T21:36:01.663846827Z","pruned":[]}
4392025-08-26T21:46:01.789ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4402025-08-26T21:46:01.790ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4412025-08-26T21:46:01.791ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
4422025-08-26T21:46:01.791ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4432025-08-26T21:46:01.792ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a42df10f-3f4b-4b08-816f-c447e6027386", "content-length": "26", "date": "Tue, 26 Aug 2025 21:46:01 GMT"} })
4442025-08-26T21:46:01.792ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4452025-08-26T21:46:01.793ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4462025-08-26T21:46:01.816ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4472025-08-26T21:46:01.821ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
4482025-08-26T21:46:01.821ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
4492025-08-26T21:46:01.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
4502025-08-26T21:46:01.822ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
4512025-08-26T21:46:01.823ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4522025-08-26T21:46:01.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
4532025-08-26T21:46:01.823ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4542025-08-26T21:46:01.823ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
4552025-08-26T21:46:01.823ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4562025-08-26T21:46:01.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
4572025-08-26T21:46:01.825ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4582025-08-26T21:46:01.826ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
4592025-08-26T21:46:01.826ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
4602025-08-26T21:46:01.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
4612025-08-26T21:46:01.828ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4622025-08-26T21:46:01.828ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
4632025-08-26T21:46:01.828ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
4642025-08-26T21:46:01.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
4652025-08-26T21:46:01.830ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4662025-08-26T21:46:01.830ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4672025-08-26T21:46:01.830ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4682025-08-26T21:46:01.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4692025-08-26T21:46:01.831ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4702025-08-26T21:46:01.833ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4712025-08-26T21:46:01.833ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
4722025-08-26T21:46:01.833ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4732025-08-26T21:46:01.833ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4742025-08-26T21:46:01.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4752025-08-26T21:46:01.834ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4762025-08-26T21:46:01.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4772025-08-26T21:46:01.835ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4782025-08-26T21:46:01.836ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4792025-08-26T21:46:01.836ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
4802025-08-26T21:46:01.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
4812025-08-26T21:46:01.838ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
4822025-08-26T21:46:01.838ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
4832025-08-26T21:46:01.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
4842025-08-26T21:46:01.839ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4852025-08-26T21:46:01.839ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4862025-08-26T21:46:01.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
4872025-08-26T21:46:01.839ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
4882025-08-26T21:46:01.842ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4892025-08-26T21:46:01.842ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
4902025-08-26T21:46:01.842ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
4912025-08-26T21:46:01.842ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
4922025-08-26T21:46:01.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
4932025-08-26T21:46:01.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
4942025-08-26T21:46:01.854ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4952025-08-26T21:46:01.854ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 168.605781ms iteration = 1 status = {"errors":[],"garbage_collect_requested":[]}
4962025-08-26T21:46:01.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4972025-08-26T21:46:01.855ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 169.114083ms iteration = 1 status = {"errors":[],"found_port_ok":[]}
4982025-08-26T21:46:01.857ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 171.2363ms iteration = 1 status = {"errors":[],"finish_invoked_ok":[]}
4992025-08-26T21:46:01.862ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
5002025-08-26T21:46:01.862ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 196.690444ms iteration = 1 status = {"chicken_switches_updated":false}
5012025-08-26T21:46:01.864ZINFOnexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:473
5022025-08-26T21:46:01.864ZINFOnexus (SagaQuiesceHandle): saga recovery pass finished file = nexus/types/src/quiesce.rs:332 generation = 1
5032025-08-26T21:46:01.864ZDEBGnexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 178.361351ms iteration = 1 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
5042025-08-26T21:46:01.864ZWARNnexus (BackgroundTasks): inventory collection is None background_task = service_zone_nat_tracker file = nexus/src/app/background/tasks/sync_service_zone_nat.rs:91
5052025-08-26T21:46:01.864ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 191.64416ms iteration = 1 status = {"error":"inventory collection is None"}
5062025-08-26T21:46:01.865ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5072025-08-26T21:46:01.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5082025-08-26T21:46:01.866ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5092025-08-26T21:46:01.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
5102025-08-26T21:46:01.870ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
5112025-08-26T21:46:01.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
5122025-08-26T21:46:01.873ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 200.5168ms iteration = 1 status = {}
5132025-08-26T21:46:01.874ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5142025-08-26T21:46:01.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5152025-08-26T21:46:01.878ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
5162025-08-26T21:46:01.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
5172025-08-26T21:46:01.899ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 236.715996ms iteration = 1 status = {}
5182025-08-26T21:46:01.901ZWARNnexus (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Internal Error: no target blueprint set file = nexus/src/app/background/tasks/blueprint_load.rs:69
5192025-08-26T21:46:01.901ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 228.152372ms iteration = 1 status = {"error":"failed to read target blueprint: Internal Error: no target blueprint set"}
5202025-08-26T21:46:01.908ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
5212025-08-26T21:46:01.908ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 865, tv_nsec: 768375564 } })
5222025-08-26T21:46:01.914ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
5232025-08-26T21:46:01.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
5242025-08-26T21:46:01.928ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 242.534147ms iteration = 1 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
5252025-08-26T21:46:01.930ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
5262025-08-26T21:46:01.930ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
5272025-08-26T21:46:01.930ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 244.383279ms iteration = 1 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
5282025-08-26T21:46:01.933ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5292025-08-26T21:46:01.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
5302025-08-26T21:46:01.936ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5312025-08-26T21:46:01.936ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
5322025-08-26T21:46:01.936ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5332025-08-26T21:46:01.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
5342025-08-26T21:46:01.939ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 253.413842ms iteration = 1 status = {"errors":[],"requests_created_ok":[]}
5352025-08-26T21:46:01.942ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 279.542932ms iteration = 1 status = null
5362025-08-26T21:46:01.943ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 256.991523ms iteration = 1 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
5372025-08-26T21:46:01.958ZWARNnexus (BackgroundTasks): failed to read DNS config background_task = dns_config_external dns_group = external error = Internal Error: expected exactly one latest version for DNS group External, found 0 file = nexus/src/app/background/tasks/dns_config.rs:72
5382025-08-26T21:46:01.958ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 294.875267ms iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group External, found 0"}
5392025-08-26T21:46:01.958ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("5362db44-dfc6-41c1-8ec8-32ab3e86b7b3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("5e2b44ef-e7e6-47ec-9087-a2fb44127ede.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("70604151-9a25-4539-bcae-b8d33d91f6a5.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("be83bf79-a7c5-40a7-be80-31ec2f206604.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("d35c8171-47d8-4704-93d7-f07739a9281c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d35c8171-47d8-4704-93d7-f07739a9281c.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::7)) }], valid_until: Instant { tv_sec: 865, tv_nsec: 818194037 } })
5402025-08-26T21:46:01.966ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5412025-08-26T21:46:01.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5422025-08-26T21:46:01.967ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
5432025-08-26T21:46:01.967ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
5442025-08-26T21:46:01.967ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 293.951276ms iteration = 1 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
5452025-08-26T21:46:01.968ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5462025-08-26T21:46:01.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5472025-08-26T21:46:01.970ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5482025-08-26T21:46:01.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5492025-08-26T21:46:01.971ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5502025-08-26T21:46:01.971ZWARNnexus (BackgroundTasks): failed to read Silo/DNS/TLS configuration background_task = external_endpoints error = Internal Error: expected at least one external DNS zone file = nexus/src/app/background/tasks/external_endpoints.rs:48
5512025-08-26T21:46:01.971ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 308.551365ms iteration = 1 status = {"error":"failed to read Silo/DNS/TLS configuration: Internal Error: expected at least one external DNS zone"}
5522025-08-26T21:46:01.972ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 287.361328ms iteration = 1 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
5532025-08-26T21:46:01.975ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
5542025-08-26T21:46:01.975ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 302.472579ms iteration = 1 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
5552025-08-26T21:46:01.976ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
5562025-08-26T21:46:01.976ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5572025-08-26T21:46:01.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5582025-08-26T21:46:01.977ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5592025-08-26T21:46:01.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5602025-08-26T21:46:01.979ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
5612025-08-26T21:46:01.979ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 293.961597ms iteration = 1 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
5622025-08-26T21:46:01.982ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5632025-08-26T21:46:01.982ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5642025-08-26T21:46:01.984ZWARNnexus (BackgroundTasks): failed to read DNS config background_task = dns_config_internal dns_group = internal error = Internal Error: expected exactly one latest version for DNS group Internal, found 0 file = nexus/src/app/background/tasks/dns_config.rs:72
5652025-08-26T21:46:01.984ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 321.363025ms iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group Internal, found 0"}
5662025-08-26T21:46:02.012ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 339.486943ms iteration = 1 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
5672025-08-26T21:46:02.030ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 344.131608ms iteration = 1 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
5682025-08-26T21:46:02.033ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5692025-08-26T21:46:02.034ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
5702025-08-26T21:46:02.037ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 351.695648ms iteration = 1 status = {}
5712025-08-26T21:46:02.039ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
5722025-08-26T21:46:02.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
5732025-08-26T21:46:02.042ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5742025-08-26T21:46:02.042ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
5752025-08-26T21:46:02.042ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 357.149931ms iteration = 1 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
5762025-08-26T21:46:02.050ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5772025-08-26T21:46:02.056ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
5782025-08-26T21:46:02.056ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5792025-08-26T21:46:02.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5802025-08-26T21:46:02.059ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
5812025-08-26T21:46:02.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
5822025-08-26T21:46:02.078ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5832025-08-26T21:46:02.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5842025-08-26T21:46:02.091ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5852025-08-26T21:46:02.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5862025-08-26T21:46:02.099ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5872025-08-26T21:46:02.102ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
5882025-08-26T21:46:02.102ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
5892025-08-26T21:46:02.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
5902025-08-26T21:46:02.103ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5912025-08-26T21:46:02.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5922025-08-26T21:46:02.117ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5932025-08-26T21:46:02.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5942025-08-26T21:46:02.119ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5952025-08-26T21:46:02.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
5962025-08-26T21:46:02.119ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
5972025-08-26T21:46:02.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
5982025-08-26T21:46:02.121ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
5992025-08-26T21:46:02.121ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6002025-08-26T21:46:02.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6012025-08-26T21:46:02.124ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
6022025-08-26T21:46:02.124ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
6032025-08-26T21:46:02.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
6042025-08-26T21:46:02.125ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6052025-08-26T21:46:02.130ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
6062025-08-26T21:46:02.134ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {} subnet_networks = {} vpc_interfaces = {}
6072025-08-26T21:46:02.134ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6082025-08-26T21:46:02.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6092025-08-26T21:46:02.137ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 451.044057ms iteration = 1 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":0,"put_config_err":0,"put_config_ok":0,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":0,"put_config_err":0,"put_config_ok":0,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[]}
6102025-08-26T21:46:02.142ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6112025-08-26T21:46:02.144ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
6122025-08-26T21:46:02.144ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
6132025-08-26T21:46:02.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
6142025-08-26T21:46:02.163ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
6152025-08-26T21:46:02.163ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = []
6162025-08-26T21:46:02.163ZDEBGnexus (service-firewall-rule-progator): resolved 0 rules for sleds background_task = service_firewall_rule_propagation
6172025-08-26T21:46:02.180ZDEBGnexus (CockroachClusterAdminClient): Updated CockroachDB cluster backends added = 5 addresses = [[fd00:1122:3344:101::6]:32222, [fd00:1122:3344:101::5]:32222, [fd00:1122:3344:101::3]:32222, [fd00:1122:3344:101::4]:32222, [fd00:1122:3344:101::7]:32222] removed = 0 total = 5
6182025-08-26T21:46:02.184ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
6192025-08-26T21:46:02.184ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
6202025-08-26T21:46:02.184ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
6212025-08-26T21:46:02.186ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "10c8fa6b-6c04-4dfb-a418-360f933ae71d", "content-length": "476", "date": "Tue, 26 Aug 2025 21:46:02 GMT"} })
6222025-08-26T21:46:02.194ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
6232025-08-26T21:46:02.195ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "34a5d1e5-3d4d-4f9e-802c-b24c8283890c", "content-length": "734", "date": "Tue, 26 Aug 2025 21:46:01 GMT"} })
6242025-08-26T21:46:02.196ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
6252025-08-26T21:46:02.197ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "16094210-8185-4297-a5ea-fa4011fd3d7a", "content-length": "111", "date": "Tue, 26 Aug 2025 21:46:01 GMT"} })
6262025-08-26T21:46:02.200ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
6272025-08-26T21:46:02.200ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5bb383aa-a40b-4733-bfe6-3188a77476b0", "content-length": "111", "date": "Tue, 26 Aug 2025 21:46:01 GMT"} })
6282025-08-26T21:46:02.200ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
6292025-08-26T21:46:02.201ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a91ac52-739f-4f3a-8af8-2a3c7598c492", "content-length": "163", "date": "Tue, 26 Aug 2025 21:46:01 GMT"} })
6302025-08-26T21:46:02.201ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
6312025-08-26T21:46:02.201ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "227848eb-bce8-40ae-a29c-bcf357d5e0d4", "content-length": "163", "date": "Tue, 26 Aug 2025 21:46:02 GMT"} })
6322025-08-26T21:46:02.201ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
6332025-08-26T21:46:02.202ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9124f201-acb3-4fce-86d5-07b507430ba3", "content-length": "166", "date": "Tue, 26 Aug 2025 21:46:02 GMT"} })
6342025-08-26T21:46:02.202ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
6352025-08-26T21:46:02.202ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc8400fc-86c4-43d0-a9e1-ed6f95016333", "content-length": "166", "date": "Tue, 26 Aug 2025 21:46:01 GMT"} })
6362025-08-26T21:46:02.204ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
6372025-08-26T21:46:02.204ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b799c3d7-09f2-4323-9298-2ea014d98440", "content-length": "702", "date": "Tue, 26 Aug 2025 21:46:01 GMT"} })
6382025-08-26T21:46:02.205ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
6392025-08-26T21:46:02.206ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "18fd0606-58f8-4b4e-8956-0e2c6607c6a1", "content-length": "727", "date": "Tue, 26 Aug 2025 21:46:01 GMT"} })
6402025-08-26T21:46:02.206ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
6412025-08-26T21:46:02.206ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "665044e0-5d0c-4a5f-bc48-d8aa83bce6bf", "content-length": "729", "date": "Tue, 26 Aug 2025 21:46:01 GMT"} })
6422025-08-26T21:46:02.206ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
6432025-08-26T21:46:02.206ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3384b4f-4899-4a6e-bf97-3b56b075cf04", "content-length": "728", "date": "Tue, 26 Aug 2025 21:46:02 GMT"} })
6442025-08-26T21:46:02.206ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
6452025-08-26T21:46:02.207ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "187688e3-fac0-491c-bfa2-9db5c58f5066", "content-length": "722", "date": "Tue, 26 Aug 2025 21:46:01 GMT"} })
6462025-08-26T21:46:02.207ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
6472025-08-26T21:46:02.207ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "7482e7e5-6cc8-49d1-a59f-061772231f41", "content-length": "10", "date": "Tue, 26 Aug 2025 21:46:01 GMT"} })
6482025-08-26T21:46:02.207ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
6492025-08-26T21:46:02.207ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d014c96-7d41-400e-a755-9d257067f0d1", "content-length": "144", "date": "Tue, 26 Aug 2025 21:46:01 GMT"} })
6502025-08-26T21:46:02.207ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
6512025-08-26T21:46:02.208ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "14078057-4fa1-49bc-97a6-de7b974752e3", "content-length": "147", "date": "Tue, 26 Aug 2025 21:46:01 GMT"} })
6522025-08-26T21:46:02.208ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
6532025-08-26T21:46:02.208ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8607bfa3-2988-41e7-ab34-3884ea0f89cf", "content-length": "109", "date": "Tue, 26 Aug 2025 21:46:02 GMT"} })
6542025-08-26T21:46:02.208ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
6552025-08-26T21:46:02.208ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c72d55b1-e2c0-49fe-aa0d-052d07b7ae1b", "content-length": "109", "date": "Tue, 26 Aug 2025 21:46:01 GMT"} })
6562025-08-26T21:46:02.208ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
6572025-08-26T21:46:02.209ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "af3f81b4-6624-4ac4-b336-72bb13156f06", "content-length": "163", "date": "Tue, 26 Aug 2025 21:46:01 GMT"} })
6582025-08-26T21:46:02.209ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
6592025-08-26T21:46:02.209ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac36b3bd-8f31-4c3d-ba0f-7be778563447", "content-length": "163", "date": "Tue, 26 Aug 2025 21:46:02 GMT"} })
6602025-08-26T21:46:02.209ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
6612025-08-26T21:46:02.210ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4ea53cd-cb55-465b-b1fe-e6430531afa7", "content-length": "166", "date": "Tue, 26 Aug 2025 21:46:02 GMT"} })
6622025-08-26T21:46:02.210ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
6632025-08-26T21:46:02.210ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b99b1fa6-3398-4dfe-825c-f37b647de001", "content-length": "166", "date": "Tue, 26 Aug 2025 21:46:02 GMT"} })
6642025-08-26T21:46:02.210ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
6652025-08-26T21:46:02.210ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "11721f66-1a88-4a4f-a0d2-3368a887b61f", "content-length": "702", "date": "Tue, 26 Aug 2025 21:46:02 GMT"} })
6662025-08-26T21:46:02.210ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
6672025-08-26T21:46:02.211ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "43bd7a86-d997-4ef8-9f63-1a0334c2bf69", "content-length": "727", "date": "Tue, 26 Aug 2025 21:46:02 GMT"} })
6682025-08-26T21:46:02.211ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
6692025-08-26T21:46:02.211ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "35e37c2e-e336-405b-b554-e1ca23023ff3", "content-length": "729", "date": "Tue, 26 Aug 2025 21:46:02 GMT"} })
6702025-08-26T21:46:02.211ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
6712025-08-26T21:46:02.211ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1c843fe-1f68-43a2-ba3c-2441a57c93ae", "content-length": "728", "date": "Tue, 26 Aug 2025 21:46:02 GMT"} })
6722025-08-26T21:46:02.211ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
6732025-08-26T21:46:02.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
6742025-08-26T21:46:02.216ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6752025-08-26T21:46:02.219ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
6762025-08-26T21:46:02.219ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
6772025-08-26T21:46:02.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
6782025-08-26T21:46:02.221ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6792025-08-26T21:46:02.224ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
6802025-08-26T21:46:02.224ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
6812025-08-26T21:46:02.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
6822025-08-26T21:46:02.228ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6832025-08-26T21:46:02.228ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
6842025-08-26T21:46:02.228ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
6852025-08-26T21:46:02.228ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
6862025-08-26T21:46:02.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6872025-08-26T21:46:02.228ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
6882025-08-26T21:46:02.228ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
6892025-08-26T21:46:02.228ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
6902025-08-26T21:46:02.228ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
6912025-08-26T21:46:02.228ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
6922025-08-26T21:46:02.230ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6932025-08-26T21:46:02.232ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
6942025-08-26T21:46:02.232ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6952025-08-26T21:46:02.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
6962025-08-26T21:46:02.235ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6972025-08-26T21:46:02.237ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
6982025-08-26T21:46:02.237ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6992025-08-26T21:46:02.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7002025-08-26T21:46:02.276ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
7012025-08-26T21:46:02.276ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 590.949711ms iteration = 1 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
7022025-08-26T21:46:02.368ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
7032025-08-26T21:46:02.372ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "912c0438-bfc9-4a7a-bbe5-20b2ca8616e7", "content-length": "691", "date": "Tue, 26 Aug 2025 21:46:02 GMT"} })
7042025-08-26T21:46:02.374ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
7052025-08-26T21:46:02.374ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
7062025-08-26T21:46:02.374ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
7072025-08-26T21:46:02.375ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 702.169138ms iteration = 1 status = {}
7082025-08-26T21:46:02.410ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = []
7092025-08-26T21:46:02.410ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
7102025-08-26T21:46:02.410ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 0 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:491
7112025-08-26T21:46:02.410ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 725.34337ms
7122025-08-26T21:46:02.410ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 725.363881ms iteration = 1 status = {}
7132025-08-26T21:46:03.294ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7142025-08-26T21:46:03.295ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7152025-08-26T21:46:03.296ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7162025-08-26T21:46:03.297ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7172025-08-26T21:46:03.299ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7182025-08-26T21:46:03.303ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: e37a46f2-f789-4ca4-9841-f8c6dd705964, lookup_type: ById(e37a46f2-f789-4ca4-9841-f8c6dd705964) } result = Ok(())
7192025-08-26T21:46:03.303ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: e37a46f2-f789-4ca4-9841-f8c6dd705964, time_created: 2025-08-26T21:46:02.666890Z, time_modified: 2025-08-26T21:46:02.666890Z }, initialized: false, tuf_base_url: None, rack_subnet: None } file = nexus/src/app/rack.rs:775
7202025-08-26T21:46:03.486ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7e99678-b5e9-4be7-ad4c-3310e076d3a8", "content-length": "64", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
7212025-08-26T21:46:03.486ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
7222025-08-26T21:46:03.497ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8db1cde3-d537-435d-94b5-4d2584bf1150", "content-length": "64", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
7232025-08-26T21:46:03.497ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
7242025-08-26T21:46:03.514ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4940ed20-08c3-43d0-ab2f-070f2e924829", "content-length": "64", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
7252025-08-26T21:46:03.514ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
7262025-08-26T21:46:03.527ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "828a9b79-4c60-4cfd-9b26-cbb59f643790", "content-length": "64", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
7272025-08-26T21:46:03.527ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
7282025-08-26T21:46:03.530ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3e8a672-8fc7-4a6e-9279-b059b2b502db", "content-length": "64", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
7292025-08-26T21:46:03.530ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
7302025-08-26T21:46:03.759ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "57dee416-6e81-4b04-ba15-478fe3f912cc", "content-length": "336579", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
7312025-08-26T21:46:03.785ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "93192a84-20ef-45f4-9d72-8d415e1d8119", "content-length": "358398", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
7322025-08-26T21:46:03.801ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "384375bf-f046-477f-aceb-f7c74b11eb8d", "content-length": "339769", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
7332025-08-26T21:46:03.905ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5fe52d4-97b0-4f59-8f17-7c13af651e4a", "content-length": "360639", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
7342025-08-26T21:46:03.932ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa651bcc-0637-4b55-b04e-53123c62709a", "content-length": "376912", "date": "Tue, 26 Aug 2025 21:46:03 GMT"} })
7352025-08-26T21:46:03.935ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
7362025-08-26T21:46:03.935ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
7372025-08-26T21:46:03.935ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
7382025-08-26T21:46:03.935ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
7392025-08-26T21:46:03.935ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
7402025-08-26T21:46:03.936ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
7412025-08-26T21:46:03.936ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
7422025-08-26T21:46:03.936ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
7432025-08-26T21:46:03.939ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
7442025-08-26T21:46:03.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
7452025-08-26T21:46:03.941ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7462025-08-26T21:46:03.944ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
7472025-08-26T21:46:03.944ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
7482025-08-26T21:46:03.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
7492025-08-26T21:46:04.497ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 0dd3b372-00f5-4ddd-acec-8c8a4f49c797 file = nexus/db-queries/src/db/datastore/inventory.rs:1614
7502025-08-26T21:46:04.497ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 0dd3b372-00f5-4ddd-acec-8c8a4f49c797 time_started = 2025-08-26 21:46:02.180933 UTC
7512025-08-26T21:46:04.497ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 2 reason = Dependency
7522025-08-26T21:46:04.497ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 2 reason = Dependency
7532025-08-26T21:46:04.497ZWARNnexus (BackgroundTasks): Blueprint rendezvous: skipped background_task = blueprint_rendezvous file = nexus/src/app/background/tasks/blueprint_rendezvous.rs:52 reason = no blueprint
7542025-08-26T21:46:04.497ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 14.3\u{b5}s iteration = 2 status = {"error":"no blueprint"}
7552025-08-26T21:46:04.497ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7562025-08-26T21:46:04.497ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 2 reason = Dependency
7572025-08-26T21:46:04.497ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
7582025-08-26T21:46:04.497ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 5.06\u{b5}s iteration = 2 status = "Disabled"
7592025-08-26T21:46:04.497ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 2.831880147s iteration = 1 status = {"collection_id":"0dd3b372-00f5-4ddd-acec-8c8a4f49c797","time_done":"2025-08-26 21:46:03.936908 UTC","time_started":"2025-08-26 21:46:02.180933 UTC"}
7602025-08-26T21:46:04.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7612025-08-26T21:46:04.499ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7622025-08-26T21:46:04.501ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
7632025-08-26T21:46:04.501ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
7642025-08-26T21:46:04.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7652025-08-26T21:46:04.503ZINFOnexus (BackgroundTasks): Physical Disk Adoption: Rack not yet initialized background_task = physical_disk_adoption file = nexus/src/app/background/tasks/physical_disk_adoption.rs:81 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
7662025-08-26T21:46:04.503ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 5.932213ms iteration = 2 status = {"error":"rack not yet initialized: e37a46f2-f789-4ca4-9841-f8c6dd705964"}
7672025-08-26T21:46:05.304ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7682025-08-26T21:46:05.304ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7692025-08-26T21:46:05.306ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7702025-08-26T21:46:05.306ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7712025-08-26T21:46:05.308ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7722025-08-26T21:46:05.311ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: e37a46f2-f789-4ca4-9841-f8c6dd705964, lookup_type: ById(e37a46f2-f789-4ca4-9841-f8c6dd705964) } result = Ok(())
7732025-08-26T21:46:05.311ZINFOnexus: Still waiting for rack initialization: Rack { identity: RackIdentity { id: e37a46f2-f789-4ca4-9841-f8c6dd705964, time_created: 2025-08-26T21:46:02.666890Z, time_modified: 2025-08-26T21:46:02.666890Z }, initialized: false, tuf_base_url: None, rack_subnet: Some(V6(Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56 })) } file = nexus/src/app/rack.rs:775
7742025-08-26T21:46:06.665ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 2 reason = Timeout
7752025-08-26T21:46:06.665ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
7762025-08-26T21:46:06.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
7772025-08-26T21:46:06.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7782025-08-26T21:46:06.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
7792025-08-26T21:46:06.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
7802025-08-26T21:46:06.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
7812025-08-26T21:46:06.685ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
7822025-08-26T21:46:06.685ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 20.312607ms iteration = 2 status = {"chicken_switches_updated":false}
7832025-08-26T21:46:07.312ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7842025-08-26T21:46:07.312ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7852025-08-26T21:46:07.314ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {} }
7862025-08-26T21:46:07.314ZDEBGnexus (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
7872025-08-26T21:46:07.316ZDEBGnexus (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7882025-08-26T21:46:07.319ZDEBGnexus (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: e37a46f2-f789-4ca4-9841-f8c6dd705964, lookup_type: ById(e37a46f2-f789-4ca4-9841-f8c6dd705964) } result = Ok(())
7892025-08-26T21:46:07.319ZINFOnexus: Rack initialized file = nexus/src/app/rack.rs:772
7902025-08-26T21:46:07.319ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
7912025-08-26T21:46:07.319ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
7922025-08-26T21:46:07.321ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
7932025-08-26T21:46:07.321ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
7942025-08-26T21:46:07.323ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
7952025-08-26T21:46:07.323ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
7962025-08-26T21:46:07.325ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
7972025-08-26T21:46:07.325ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
7982025-08-26T21:46:07.332ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
7992025-08-26T21:46:07.332ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8002025-08-26T21:46:07.333ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8012025-08-26T21:46:07.333ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8022025-08-26T21:46:07.335ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8032025-08-26T21:46:07.338ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
8042025-08-26T21:46:07.338ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8052025-08-26T21:46:07.338ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8062025-08-26T21:46:07.340ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8072025-08-26T21:46:07.340ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8082025-08-26T21:46:07.341ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8092025-08-26T21:46:07.341ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8102025-08-26T21:46:07.342ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8112025-08-26T21:46:07.346ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
8122025-08-26T21:46:07.346ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8132025-08-26T21:46:07.346ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8142025-08-26T21:46:07.357ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8152025-08-26T21:46:07.357ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8162025-08-26T21:46:07.359ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8172025-08-26T21:46:07.359ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8182025-08-26T21:46:07.360ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8192025-08-26T21:46:07.360ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8202025-08-26T21:46:07.362ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8212025-08-26T21:46:07.362ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8222025-08-26T21:46:07.363ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8232025-08-26T21:46:07.363ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8242025-08-26T21:46:07.365ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8252025-08-26T21:46:07.365ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8262025-08-26T21:46:07.366ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8272025-08-26T21:46:07.369ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
8282025-08-26T21:46:07.373ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8292025-08-26T21:46:07.373ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8302025-08-26T21:46:07.375ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8312025-08-26T21:46:07.375ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8322025-08-26T21:46:07.376ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8332025-08-26T21:46:07.376ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8342025-08-26T21:46:07.379ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8352025-08-26T21:46:07.379ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8362025-08-26T21:46:07.381ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8372025-08-26T21:46:07.381ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8382025-08-26T21:46:07.383ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8392025-08-26T21:46:07.383ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8402025-08-26T21:46:07.385ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8412025-08-26T21:46:07.385ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8422025-08-26T21:46:07.386ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8432025-08-26T21:46:07.390ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
8442025-08-26T21:46:07.390ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8452025-08-26T21:46:07.390ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8462025-08-26T21:46:07.392ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8472025-08-26T21:46:07.392ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8482025-08-26T21:46:07.394ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8492025-08-26T21:46:07.394ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8502025-08-26T21:46:07.396ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8512025-08-26T21:46:07.400ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
8522025-08-26T21:46:07.400ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8532025-08-26T21:46:07.400ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8542025-08-26T21:46:07.406ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8552025-08-26T21:46:07.406ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8562025-08-26T21:46:07.408ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8572025-08-26T21:46:07.408ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8582025-08-26T21:46:07.410ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8592025-08-26T21:46:07.410ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8602025-08-26T21:46:07.413ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8612025-08-26T21:46:07.413ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8622025-08-26T21:46:07.415ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8632025-08-26T21:46:07.415ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8642025-08-26T21:46:07.417ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8652025-08-26T21:46:07.417ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8662025-08-26T21:46:07.418ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8672025-08-26T21:46:07.418ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8682025-08-26T21:46:07.420ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8692025-08-26T21:46:07.424ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
8702025-08-26T21:46:07.424ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8712025-08-26T21:46:07.424ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8722025-08-26T21:46:07.425ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8732025-08-26T21:46:07.425ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8742025-08-26T21:46:07.427ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8752025-08-26T21:46:07.427ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8762025-08-26T21:46:07.428ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8772025-08-26T21:46:07.433ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
8782025-08-26T21:46:07.433ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8792025-08-26T21:46:07.433ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8802025-08-26T21:46:07.442ZDEBGnexus (InternalApi): resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 0802405a-a5f2-402c-94da-ccafb58bccb5, kind: Service { id: 4996dcc4-c530-4f64-b407-fa372462bd9e }, name: Name("nexus-4996dcc4-c530-4f64-b407-fa372462bd9e"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 1acb6bfd-558b-4271-bbf5-58e0d089b710, kind: Service { id: 0205eaf0-3fc6-4ad5-b09b-732af233e9c1 }, name: Name("nexus-0205eaf0-3fc6-4ad5-b09b-732af233e9c1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 142])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cbb99fa2-edc6-497d-9c47-8dd2642f8945, kind: Service { id: 5d0959e5-47a6-4bf3-a52b-57950b9d900d }, name: Name("nexus-5d0959e5-47a6-4bf3-a52b-57950b9d900d"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 80])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 0106180c-e6f8-4d65-969e-4fbbddf596a1, kind: Service { id: 60abf731-31e1-4111-a3e2-65da89eaed87 }, name: Name("external-dns-60abf731-31e1-4111-a3e2-65da89eaed87"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 81])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b69ae953-8f67-4cb3-9521-db9e707a6f2d, kind: Service { id: 84c849eb-0949-41d9-ad02-9a5509144564 }, name: Name("external-dns-84c849eb-0949-41d9-ad02-9a5509144564"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 136])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
8812025-08-26T21:46:07.442ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8822025-08-26T21:46:07.442ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8832025-08-26T21:46:07.444ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8842025-08-26T21:46:07.446ZDEBGnexus (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
8852025-08-26T21:46:07.446ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8862025-08-26T21:46:07.446ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8872025-08-26T21:46:07.448ZDEBGnexus (InternalApi): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified.
8882025-08-26T21:46:07.449ZDEBGnexus (InternalApi): resolved firewall rules for sled agents sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0106180c-e6f8-4d65-969e-4fbbddf596a1, kind: Service { id: 60abf731-31e1-4111-a3e2-65da89eaed87 }, name: Name("external-dns-60abf731-31e1-4111-a3e2-65da89eaed87"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 81])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b69ae953-8f67-4cb3-9521-db9e707a6f2d, kind: Service { id: 84c849eb-0949-41d9-ad02-9a5509144564 }, name: Name("external-dns-84c849eb-0949-41d9-ad02-9a5509144564"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 136])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0802405a-a5f2-402c-94da-ccafb58bccb5, kind: Service { id: 4996dcc4-c530-4f64-b407-fa372462bd9e }, name: Name("nexus-4996dcc4-c530-4f64-b407-fa372462bd9e"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 1acb6bfd-558b-4271-bbf5-58e0d089b710, kind: Service { id: 0205eaf0-3fc6-4ad5-b09b-732af233e9c1 }, name: Name("nexus-0205eaf0-3fc6-4ad5-b09b-732af233e9c1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 142])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cbb99fa2-edc6-497d-9c47-8dd2642f8945, kind: Service { id: 5d0959e5-47a6-4bf3-a52b-57950b9d900d }, name: Name("nexus-5d0959e5-47a6-4bf3-a52b-57950b9d900d"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 80])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 0106180c-e6f8-4d65-969e-4fbbddf596a1, kind: Service { id: 60abf731-31e1-4111-a3e2-65da89eaed87 }, name: Name("external-dns-60abf731-31e1-4111-a3e2-65da89eaed87"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 81])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b69ae953-8f67-4cb3-9521-db9e707a6f2d, kind: Service { id: 84c849eb-0949-41d9-ad02-9a5509144564 }, name: Name("external-dns-84c849eb-0949-41d9-ad02-9a5509144564"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 136])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0802405a-a5f2-402c-94da-ccafb58bccb5, kind: Service { id: 4996dcc4-c530-4f64-b407-fa372462bd9e }, name: Name("nexus-4996dcc4-c530-4f64-b407-fa372462bd9e"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 1acb6bfd-558b-4271-bbf5-58e0d089b710, kind: Service { id: 0205eaf0-3fc6-4ad5-b09b-732af233e9c1 }, name: Name("nexus-0205eaf0-3fc6-4ad5-b09b-732af233e9c1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 142])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cbb99fa2-edc6-497d-9c47-8dd2642f8945, kind: Service { id: 5d0959e5-47a6-4bf3-a52b-57950b9d900d }, name: Name("nexus-5d0959e5-47a6-4bf3-a52b-57950b9d900d"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 80])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
8892025-08-26T21:46:07.449ZDEBGnexus (InternalApi): resolved 3 rules for sleds
8902025-08-26T21:46:07.477ZDEBGnexus (InternalApi): resolved sleds for vpc oxide-services vpc_to_sled = [Sled { identity: SledIdentity { id: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, time_created: 2025-08-26T21:46:02.113579Z, time_modified: 2025-08-26T21:46:02.113579Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
8912025-08-26T21:46:07.477ZDEBGnexus (InternalApi): sending firewall rules to sled agents
8922025-08-26T21:46:07.511ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8932025-08-26T21:46:07.512ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8942025-08-26T21:46:07.515ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {} }
8952025-08-26T21:46:07.515ZDEBGnexus (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
8962025-08-26T21:46:07.516ZDEBGnexus (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8972025-08-26T21:46:07.519ZDEBGnexus (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
8982025-08-26T21:46:07.519ZDEBGnexus (InternalApi): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
8992025-08-26T21:46:07.520ZDEBGnexus (InternalApi): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "37417d1e-fb8b-4201-a368-37bb42a235e2", "date": "Tue, 26 Aug 2025 21:46:06 GMT"} })
9002025-08-26T21:46:07.521ZINFOnexus (InternalApi): updated firewall rules on 1 sleds file = nexus/networking/src/firewall_rules.rs:491
9012025-08-26T21:46:07.521ZINFOnexus: plumbed initial IP allowlist file = nexus/src/app/allow_list.rs:161
9022025-08-26T21:46:11.665ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 3 reason = Timeout
9032025-08-26T21:46:11.665ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
9042025-08-26T21:46:11.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
9052025-08-26T21:46:11.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9062025-08-26T21:46:11.670ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
9072025-08-26T21:46:11.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
9082025-08-26T21:46:11.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
9092025-08-26T21:46:11.672ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
9102025-08-26T21:46:11.672ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.350115ms iteration = 3 status = {"chicken_switches_updated":false}
9112025-08-26T21:46:11.673ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 2 reason = Timeout
9122025-08-26T21:46:11.673ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9132025-08-26T21:46:11.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9142025-08-26T21:46:11.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9152025-08-26T21:46:11.678ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9162025-08-26T21:46:11.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9172025-08-26T21:46:11.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9182025-08-26T21:46:11.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9192025-08-26T21:46:11.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9202025-08-26T21:46:11.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9212025-08-26T21:46:11.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
9222025-08-26T21:46:11.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9232025-08-26T21:46:11.685ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 2 reason = Timeout
9242025-08-26T21:46:11.685ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9252025-08-26T21:46:11.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
9262025-08-26T21:46:11.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9272025-08-26T21:46:11.735ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9282025-08-26T21:46:11.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9292025-08-26T21:46:11.747ZINFOnexus (BackgroundTasks): found new target blueprint (first find) background_task = blueprint_loader file = nexus/src/app/background/tasks/blueprint_load.rs:88 target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
9302025-08-26T21:46:11.747ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 73.479837ms iteration = 2 status = {"enabled":false,"status":"first target blueprint","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z","time_found":"2025-08-26T21:46:11.747096793Z"}
9312025-08-26T21:46:11.747ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 2 reason = Dependency
9322025-08-26T21:46:11.747ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9332025-08-26T21:46:11.747ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 2 reason = Dependency
9342025-08-26T21:46:11.747ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 3 reason = Dependency
9352025-08-26T21:46:11.747ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
9362025-08-26T21:46:11.747ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
9372025-08-26T21:46:11.747ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 13.411\u{b5}s iteration = 3 status = "Disabled"
9382025-08-26T21:46:11.747ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 18.181\u{b5}s iteration = 2 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
9392025-08-26T21:46:11.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9402025-08-26T21:46:11.747ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9412025-08-26T21:46:11.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9422025-08-26T21:46:11.747ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9432025-08-26T21:46:11.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9442025-08-26T21:46:11.748ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9452025-08-26T21:46:11.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9462025-08-26T21:46:11.748ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9472025-08-26T21:46:11.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9482025-08-26T21:46:11.749ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9492025-08-26T21:46:11.751ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
9502025-08-26T21:46:11.751ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9512025-08-26T21:46:11.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9522025-08-26T21:46:11.753ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9532025-08-26T21:46:11.755ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
9542025-08-26T21:46:11.755ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9552025-08-26T21:46:11.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9562025-08-26T21:46:11.756ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9572025-08-26T21:46:11.757ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
9582025-08-26T21:46:11.757ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9592025-08-26T21:46:11.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9602025-08-26T21:46:11.758ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9612025-08-26T21:46:11.759ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
9622025-08-26T21:46:11.759ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9632025-08-26T21:46:11.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9642025-08-26T21:46:11.760ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9652025-08-26T21:46:11.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
9662025-08-26T21:46:11.761ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9672025-08-26T21:46:11.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9682025-08-26T21:46:11.776ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 91.055282ms iteration = 2 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9692025-08-26T21:46:11.797ZDEBGnexus (BackgroundTasks): client request background_task = crdb_node_id_collector body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
9702025-08-26T21:46:11.830ZDEBGnexus (BackgroundTasks): client request background_task = crdb_node_id_collector body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
9712025-08-26T21:46:11.864ZDEBGnexus (BackgroundTasks): client request background_task = crdb_node_id_collector body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
9722025-08-26T21:46:11.898ZDEBGnexus (BackgroundTasks): client request background_task = crdb_node_id_collector body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
9732025-08-26T21:46:11.933ZDEBGnexus (BackgroundTasks): client request background_task = crdb_node_id_collector body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
9742025-08-26T21:46:11.933ZDEBGnexus (BackgroundTasks): client response background_task = crdb_node_id_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f6bc23c-dea0-461c-a515-962ba54c6e1d", "content-length": "64", "date": "Tue, 26 Aug 2025 21:46:11 GMT"} })
9752025-08-26T21:46:11.933ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9762025-08-26T21:46:11.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9772025-08-26T21:46:11.934ZDEBGnexus (BackgroundTasks): client response background_task = crdb_node_id_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "22212171-87d0-4332-89d6-72b812a1fb37", "content-length": "64", "date": "Tue, 26 Aug 2025 21:46:11 GMT"} })
9782025-08-26T21:46:11.934ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9792025-08-26T21:46:11.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9802025-08-26T21:46:11.934ZDEBGnexus (BackgroundTasks): client response background_task = crdb_node_id_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd1788b5-ea5d-40aa-8edc-73834e11fea8", "content-length": "64", "date": "Tue, 26 Aug 2025 21:46:11 GMT"} })
9812025-08-26T21:46:11.934ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9822025-08-26T21:46:11.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9832025-08-26T21:46:11.934ZDEBGnexus (BackgroundTasks): client response background_task = crdb_node_id_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "df0993f9-cee0-4b59-a5a1-d84193b3b1d8", "content-length": "64", "date": "Tue, 26 Aug 2025 21:46:11 GMT"} })
9842025-08-26T21:46:11.934ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9852025-08-26T21:46:11.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9862025-08-26T21:46:11.935ZDEBGnexus (BackgroundTasks): client response background_task = crdb_node_id_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6dfac81d-df15-4b5c-9089-4288a7da7a37", "content-length": "64", "date": "Tue, 26 Aug 2025 21:46:11 GMT"} })
9872025-08-26T21:46:11.935ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9882025-08-26T21:46:11.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9892025-08-26T21:46:11.936ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9902025-08-26T21:46:11.938ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
9912025-08-26T21:46:11.938ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9922025-08-26T21:46:11.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9932025-08-26T21:46:11.938ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9942025-08-26T21:46:11.939ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
9952025-08-26T21:46:11.939ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
9962025-08-26T21:46:11.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
9972025-08-26T21:46:11.939ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9982025-08-26T21:46:11.941ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
9992025-08-26T21:46:11.941ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
10002025-08-26T21:46:11.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
10012025-08-26T21:46:11.941ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10022025-08-26T21:46:11.942ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
10032025-08-26T21:46:11.942ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
10042025-08-26T21:46:11.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
10052025-08-26T21:46:11.942ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10062025-08-26T21:46:11.943ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
10072025-08-26T21:46:11.943ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
10082025-08-26T21:46:11.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
10092025-08-26T21:46:11.955ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 208.821087ms iteration = 2 status = {"nsuccess":5}
10102025-08-26T21:46:15.347ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46597 req_id = 1e25aedd-30ce-4d4b-8ff9-02e030ba5ddc roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
10112025-08-26T21:46:15.348ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46597 req_id = 1e25aedd-30ce-4d4b-8ff9-02e030ba5ddc resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
10122025-08-26T21:46:15.353ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 9103 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46597 req_id = 1e25aedd-30ce-4d4b-8ff9-02e030ba5ddc response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
10132025-08-26T21:46:15.356ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46597 req_id = 46cb16bf-fdfd-4c7c-8579-711d5a504db3 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMGJkNjIxMC00NzEyLTQ1NTMtOTg1NC05NjdkODIzNGU4MTUifX0%3D
10142025-08-26T21:46:15.356ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46597 req_id = 46cb16bf-fdfd-4c7c-8579-711d5a504db3 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMGJkNjIxMC00NzEyLTQ1NTMtOTg1NC05NjdkODIzNGU4MTUifX0%3D
10152025-08-26T21:46:15.365ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 8974 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46597 req_id = 46cb16bf-fdfd-4c7c-8579-711d5a504db3 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkMGJkNjIxMC00NzEyLTQ1NTMtOTg1NC05NjdkODIzNGU4MTUifX0%3D
10162025-08-26T21:46:16.665ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 4 reason = Timeout
10172025-08-26T21:46:16.665ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
10182025-08-26T21:46:16.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
10192025-08-26T21:46:16.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10202025-08-26T21:46:16.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
10212025-08-26T21:46:16.673ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
10222025-08-26T21:46:16.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
10232025-08-26T21:46:16.676ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
10242025-08-26T21:46:16.676ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 10.895165ms iteration = 4 status = {"chicken_switches_updated":false}
10252025-08-26T21:46:21.665ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 5 reason = Timeout
10262025-08-26T21:46:21.665ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
10272025-08-26T21:46:21.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
10282025-08-26T21:46:21.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10292025-08-26T21:46:21.670ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
10302025-08-26T21:46:21.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
10312025-08-26T21:46:21.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
10322025-08-26T21:46:21.672ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
10332025-08-26T21:46:21.672ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.151971ms iteration = 5 status = {"chicken_switches_updated":false}
10342025-08-26T21:46:21.673ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 3 reason = Timeout
10352025-08-26T21:46:21.673ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10362025-08-26T21:46:21.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10372025-08-26T21:46:21.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10382025-08-26T21:46:21.678ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
10392025-08-26T21:46:21.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10402025-08-26T21:46:21.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10412025-08-26T21:46:21.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10422025-08-26T21:46:21.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10432025-08-26T21:46:21.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10442025-08-26T21:46:21.685ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 3 reason = Timeout
10452025-08-26T21:46:21.685ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10462025-08-26T21:46:21.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10472025-08-26T21:46:21.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
10482025-08-26T21:46:21.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10492025-08-26T21:46:21.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
10502025-08-26T21:46:21.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10512025-08-26T21:46:21.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
10522025-08-26T21:46:21.688ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.442008ms iteration = 3 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
10532025-08-26T21:46:21.704ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
10542025-08-26T21:46:21.704ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 30.61098ms iteration = 3 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
10552025-08-26T21:46:26.665ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 6 reason = Timeout
10562025-08-26T21:46:26.665ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
10572025-08-26T21:46:26.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
10582025-08-26T21:46:26.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10592025-08-26T21:46:26.670ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
10602025-08-26T21:46:26.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
10612025-08-26T21:46:26.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
10622025-08-26T21:46:26.672ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
10632025-08-26T21:46:26.672ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.358225ms iteration = 6 status = {"chicken_switches_updated":false}
10642025-08-26T21:46:31.663ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 2 reason = Timeout
10652025-08-26T21:46:31.663ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 2 reason = Timeout
10662025-08-26T21:46:31.663ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 2 reason = Timeout
10672025-08-26T21:46:31.663ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
10682025-08-26T21:46:31.663ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
10692025-08-26T21:46:31.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
10702025-08-26T21:46:31.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
10712025-08-26T21:46:31.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 7 reason = Timeout
10722025-08-26T21:46:31.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
10732025-08-26T21:46:31.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
10742025-08-26T21:46:31.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10752025-08-26T21:46:31.668ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 895, tv_nsec: 527599956 } })
10762025-08-26T21:46:31.668ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
10772025-08-26T21:46:31.670ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
10782025-08-26T21:46:31.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
10792025-08-26T21:46:31.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
10802025-08-26T21:46:31.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
10812025-08-26T21:46:31.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.360216ms iteration = 7 status = {"chicken_switches_updated":false}
10822025-08-26T21:46:31.673ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 2 reason = Timeout
10832025-08-26T21:46:31.673ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 2 reason = Timeout
10842025-08-26T21:46:31.673ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 3 reason = Timeout
10852025-08-26T21:46:31.673ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
10862025-08-26T21:46:31.673ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10872025-08-26T21:46:31.673ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10882025-08-26T21:46:31.673ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 2 reason = Timeout
10892025-08-26T21:46:31.673ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10902025-08-26T21:46:31.673ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 2 reason = Timeout
10912025-08-26T21:46:31.673ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 4 reason = Timeout
10922025-08-26T21:46:31.673ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 2 reason = Timeout
10932025-08-26T21:46:31.673ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
10942025-08-26T21:46:31.673ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10952025-08-26T21:46:31.673ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10962025-08-26T21:46:31.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
10972025-08-26T21:46:31.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10982025-08-26T21:46:31.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10992025-08-26T21:46:31.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11002025-08-26T21:46:31.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
11012025-08-26T21:46:31.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11022025-08-26T21:46:31.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11032025-08-26T21:46:31.676ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 895, tv_nsec: 535586806 } })
11042025-08-26T21:46:31.676ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
11052025-08-26T21:46:31.677ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11062025-08-26T21:46:31.677ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11072025-08-26T21:46:31.677ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11082025-08-26T21:46:31.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11092025-08-26T21:46:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
11102025-08-26T21:46:31.681ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11112025-08-26T21:46:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
11122025-08-26T21:46:31.681ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
11132025-08-26T21:46:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
11142025-08-26T21:46:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11152025-08-26T21:46:31.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11162025-08-26T21:46:31.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11172025-08-26T21:46:31.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
11182025-08-26T21:46:31.682ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11192025-08-26T21:46:31.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11202025-08-26T21:46:31.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11212025-08-26T21:46:31.684ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 895, tv_nsec: 544149099 } })
11222025-08-26T21:46:31.684ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
11232025-08-26T21:46:31.685ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11242025-08-26T21:46:31.685ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 2 reason = Timeout
11252025-08-26T21:46:31.685ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 4 reason = Timeout
11262025-08-26T21:46:31.685ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
11272025-08-26T21:46:31.685ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
11282025-08-26T21:46:31.685ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
11292025-08-26T21:46:31.685ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 2 reason = Timeout
11302025-08-26T21:46:31.685ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
11312025-08-26T21:46:31.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11322025-08-26T21:46:31.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
11332025-08-26T21:46:31.686ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 2 reason = Timeout
11342025-08-26T21:46:31.686ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 2 reason = Timeout
11352025-08-26T21:46:31.686ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
11362025-08-26T21:46:31.686ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 14.891\u{b5}s iteration = 2 status = {"disabled":true,"errors":[],"sps":[]}
11372025-08-26T21:46:31.686ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 2 reason = Timeout
11382025-08-26T21:46:31.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
11392025-08-26T21:46:31.686ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 2 reason = Timeout
11402025-08-26T21:46:31.686ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 2 reason = Timeout
11412025-08-26T21:46:31.686ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
11422025-08-26T21:46:31.686ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
11432025-08-26T21:46:31.686ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
11442025-08-26T21:46:31.686ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 2 reason = Timeout
11452025-08-26T21:46:31.686ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
11462025-08-26T21:46:31.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
11472025-08-26T21:46:31.686ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 2 reason = Timeout
11482025-08-26T21:46:31.686ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
11492025-08-26T21:46:31.686ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11502025-08-26T21:46:31.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
11512025-08-26T21:46:31.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
11522025-08-26T21:46:31.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
11532025-08-26T21:46:31.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
11542025-08-26T21:46:31.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
11552025-08-26T21:46:31.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11562025-08-26T21:46:31.689ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 25.993436ms iteration = 2 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
11572025-08-26T21:46:31.691ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11582025-08-26T21:46:31.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11592025-08-26T21:46:31.694ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11602025-08-26T21:46:31.694ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11612025-08-26T21:46:31.698ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
11622025-08-26T21:46:31.698ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11632025-08-26T21:46:31.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11642025-08-26T21:46:31.699ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
11652025-08-26T21:46:31.699ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11662025-08-26T21:46:31.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
11672025-08-26T21:46:31.736ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
11682025-08-26T21:46:31.737ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11692025-08-26T21:46:31.737ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c054776-0a38-4812-bfb7-98dc4a8f8a17", "content-length": "26", "date": "Tue, 26 Aug 2025 21:46:31 GMT"} })
11702025-08-26T21:46:31.737ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11712025-08-26T21:46:31.739ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
11722025-08-26T21:46:31.750ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
11732025-08-26T21:46:31.751ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 77.564078ms iteration = 4 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
11742025-08-26T21:46:31.751ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
11752025-08-26T21:46:31.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
11762025-08-26T21:46:31.753ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11772025-08-26T21:46:31.755ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 81.717942ms iteration = 2 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
11782025-08-26T21:46:31.756ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
11792025-08-26T21:46:31.756ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11802025-08-26T21:46:31.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11812025-08-26T21:46:31.759ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11822025-08-26T21:46:31.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11832025-08-26T21:46:31.760ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11842025-08-26T21:46:31.761ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11852025-08-26T21:46:31.764ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
11862025-08-26T21:46:31.764ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
11872025-08-26T21:46:31.764ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6c1bd09-92dd-4216-8000-8ac290986881", "content-length": "26", "date": "Tue, 26 Aug 2025 21:46:31 GMT"} })
11882025-08-26T21:46:31.764ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
11892025-08-26T21:46:31.764ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
11902025-08-26T21:46:31.764ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11912025-08-26T21:46:31.764ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
11922025-08-26T21:46:31.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11932025-08-26T21:46:31.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
11942025-08-26T21:46:31.765ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
11952025-08-26T21:46:31.765ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
11962025-08-26T21:46:31.765ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11972025-08-26T21:46:31.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11982025-08-26T21:46:31.766ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11992025-08-26T21:46:31.770ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
12002025-08-26T21:46:31.770ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
12012025-08-26T21:46:31.770ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 85.094168ms iteration = 4 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
12022025-08-26T21:46:31.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
12032025-08-26T21:46:31.773ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 87.446921ms iteration = 2 status = {"errors":[],"garbage_collect_requested":[]}
12042025-08-26T21:46:31.773ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
12052025-08-26T21:46:31.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
12062025-08-26T21:46:31.774ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 88.676909ms iteration = 2 status = {"errors":[],"finish_invoked_ok":[]}
12072025-08-26T21:46:31.776ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
12082025-08-26T21:46:31.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
12092025-08-26T21:46:31.778ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
12102025-08-26T21:46:31.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
12112025-08-26T21:46:31.779ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 92.935365ms iteration = 2 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
12122025-08-26T21:46:31.779ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
12132025-08-26T21:46:31.779ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 93.396486ms iteration = 2 status = {"errors":[],"requests_created_ok":[]}
12142025-08-26T21:46:31.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
12152025-08-26T21:46:31.779ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12162025-08-26T21:46:31.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12172025-08-26T21:46:31.780ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12182025-08-26T21:46:31.781ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
12192025-08-26T21:46:31.781ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
12202025-08-26T21:46:31.781ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
12212025-08-26T21:46:31.781ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 95.467932ms iteration = 2 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
12222025-08-26T21:46:31.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
12232025-08-26T21:46:31.783ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
12242025-08-26T21:46:31.783ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
12252025-08-26T21:46:31.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
12262025-08-26T21:46:31.792ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12272025-08-26T21:46:31.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12282025-08-26T21:46:31.796ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12292025-08-26T21:46:31.798ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
12302025-08-26T21:46:31.798ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12312025-08-26T21:46:31.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12322025-08-26T21:46:31.802ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12332025-08-26T21:46:31.805ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
12342025-08-26T21:46:31.805ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12352025-08-26T21:46:31.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12362025-08-26T21:46:31.830ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
12372025-08-26T21:46:31.830ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12382025-08-26T21:46:31.830ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "eeeb0ac7-6e22-4f84-b9ec-e8f99bb6f0c2", "content-length": "26", "date": "Tue, 26 Aug 2025 21:46:31 GMT"} })
12392025-08-26T21:46:31.830ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12402025-08-26T21:46:31.832ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
12412025-08-26T21:46:31.832ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
12422025-08-26T21:46:31.832ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 159.376603ms iteration = 2 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
12432025-08-26T21:46:31.832ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
12442025-08-26T21:46:31.832ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 147.265639ms iteration = 2 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
12452025-08-26T21:46:31.846ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
12462025-08-26T21:46:31.846ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
12472025-08-26T21:46:31.846ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
12482025-08-26T21:46:31.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
12492025-08-26T21:46:31.879ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
12502025-08-26T21:46:31.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
12512025-08-26T21:46:31.882ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 218.501005ms iteration = 2 status = null
12522025-08-26T21:46:31.894ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
12532025-08-26T21:46:31.894ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
12542025-08-26T21:46:31.894ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 208.987751ms iteration = 2 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
12552025-08-26T21:46:31.899ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 235.751425ms iteration = 2 status = {}
12562025-08-26T21:46:31.901ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
12572025-08-26T21:46:31.901ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
12582025-08-26T21:46:31.902ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc8a220c-becd-41fb-b848-5143b623e7f6", "content-length": "372", "date": "Tue, 26 Aug 2025 21:46:31 GMT"} })
12592025-08-26T21:46:31.910ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12602025-08-26T21:46:31.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12612025-08-26T21:46:31.926ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 253.591737ms iteration = 3 status = {"physical_disks_added":0}
12622025-08-26T21:46:31.950ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12632025-08-26T21:46:31.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12642025-08-26T21:46:31.953ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12652025-08-26T21:46:31.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12662025-08-26T21:46:31.954ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12672025-08-26T21:46:31.955ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
12682025-08-26T21:46:31.955ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca2f83a2-8f16-435e-bc46-e740951b1957", "content-length": "691", "date": "Tue, 26 Aug 2025 21:46:31 GMT"} })
12692025-08-26T21:46:31.955ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
12702025-08-26T21:46:31.955ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
12712025-08-26T21:46:31.955ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
12722025-08-26T21:46:31.957ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 283.674354ms iteration = 2 status = {}
12732025-08-26T21:46:31.957ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
12742025-08-26T21:46:31.957ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12752025-08-26T21:46:31.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12762025-08-26T21:46:31.961ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 288.495654ms iteration = 2 status = 0
12772025-08-26T21:46:31.986ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12782025-08-26T21:46:31.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12792025-08-26T21:46:31.988ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
12802025-08-26T21:46:31.988ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12812025-08-26T21:46:31.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12822025-08-26T21:46:32.030ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
12832025-08-26T21:46:32.032ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
12842025-08-26T21:46:32.034ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
12852025-08-26T21:46:32.034ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
12862025-08-26T21:46:32.034ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
12872025-08-26T21:46:32.034ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
12882025-08-26T21:46:32.034ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
12892025-08-26T21:46:32.034ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
12902025-08-26T21:46:32.035ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
12912025-08-26T21:46:32.036ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
12922025-08-26T21:46:32.036ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
12932025-08-26T21:46:32.036ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
12942025-08-26T21:46:32.036ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
12952025-08-26T21:46:32.048ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
12962025-08-26T21:46:32.048ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
12972025-08-26T21:46:32.236ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "380bc541-a126-4e4d-a56c-80a70fbab313", "date": "Tue, 26 Aug 2025 21:46:32 GMT"} })
12982025-08-26T21:46:32.236ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
12992025-08-26T21:46:32.236ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
13002025-08-26T21:46:32.244ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "9483dd97-5bef-4aa7-ba50-2f56e7614b96", "content-length": "539", "date": "Tue, 26 Aug 2025 21:46:31 GMT"} })
13012025-08-26T21:46:32.246ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13022025-08-26T21:46:32.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13032025-08-26T21:46:32.267ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13042025-08-26T21:46:32.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13052025-08-26T21:46:32.286ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13062025-08-26T21:46:32.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13072025-08-26T21:46:32.288ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13082025-08-26T21:46:32.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13092025-08-26T21:46:32.290ZWARNnexus (BackgroundTasks): no bootstore config found in db background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1223 key = network_key rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
13102025-08-26T21:46:32.290ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13112025-08-26T21:46:32.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13122025-08-26T21:46:32.303ZINFOnexus (BackgroundTasks): updating bootstore config background_task = switch_port_config_manager config = EarlyNetworkConfig { body: EarlyNetworkConfigBody { ntp_servers: ["0.pool.ntp.org"], rack_network_config: Some(RackNetworkConfigV2 { bfd: [], bgp: [], infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [PortConfigV2 { addresses: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], autoneg: false, bgp_peers: [], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", routes: [RouteConfig { destination: V4(Ipv4Net { addr: 0.0.0.0, width: 0 }), nexthop: 10.151.2.100, rib_priority: None, vlan_id: None }], switch: Switch0, tx_eq: None, uplink_port_fec: Some(None), uplink_port_speed: Speed40G }], rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56 } }) }, generation: 2, schema_version: 2 } file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1280 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
13132025-08-26T21:46:32.303ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = PUT rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
13142025-08-26T21:46:32.307ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 204, headers: {"x-request-id": "0db60261-15e3-432c-979e-a0a45b5e963a", "date": "Tue, 26 Aug 2025 21:46:31 GMT"} })
13152025-08-26T21:46:32.311ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13162025-08-26T21:46:32.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13172025-08-26T21:46:32.328ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 654.570055ms iteration = 2 status = {}
13182025-08-26T21:46:36.665ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 8 reason = Timeout
13192025-08-26T21:46:36.665ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
13202025-08-26T21:46:36.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
13212025-08-26T21:46:36.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13222025-08-26T21:46:36.669ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
13232025-08-26T21:46:36.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
13242025-08-26T21:46:36.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
13252025-08-26T21:46:36.671ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
13262025-08-26T21:46:36.671ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.63401ms iteration = 8 status = {"chicken_switches_updated":false}
13272025-08-26T21:46:41.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 9 reason = Timeout
13282025-08-26T21:46:41.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
13292025-08-26T21:46:41.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
13302025-08-26T21:46:41.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13312025-08-26T21:46:41.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
13322025-08-26T21:46:41.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
13332025-08-26T21:46:41.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
13342025-08-26T21:46:41.673ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 5 reason = Timeout
13352025-08-26T21:46:41.673ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13362025-08-26T21:46:41.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
13372025-08-26T21:46:41.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.310715ms iteration = 9 status = {"chicken_switches_updated":false}
13382025-08-26T21:46:41.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13392025-08-26T21:46:41.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13402025-08-26T21:46:41.678ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13412025-08-26T21:46:41.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13422025-08-26T21:46:41.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13432025-08-26T21:46:41.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13442025-08-26T21:46:41.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13452025-08-26T21:46:41.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13462025-08-26T21:46:41.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
13472025-08-26T21:46:41.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13482025-08-26T21:46:41.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13492025-08-26T21:46:41.685ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 5 reason = Timeout
13502025-08-26T21:46:41.685ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13512025-08-26T21:46:41.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13522025-08-26T21:46:41.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13532025-08-26T21:46:41.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13542025-08-26T21:46:41.688ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.427807ms iteration = 5 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
13552025-08-26T21:46:41.803ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
13562025-08-26T21:46:41.803ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 129.855087ms iteration = 5 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
13572025-08-26T21:46:41.930ZERROnexus (BackgroundTasks): failed to produce EIP Internet Gateway mappings for sled background_task = vpc_route_manager err = NotFound { message: MessagePair { external_message: "unable to find IGW mappings for sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa: no inbound stream connection", internal_context: "" } } file = nexus/src/app/background/tasks/vpc_routes.rs:134 sled = sock
13582025-08-26T21:46:41.932ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13592025-08-26T21:46:41.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13602025-08-26T21:46:41.935ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13612025-08-26T21:46:41.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13622025-08-26T21:46:41.940ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13632025-08-26T21:46:41.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13642025-08-26T21:46:41.944ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
13652025-08-26T21:46:41.944ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
13662025-08-26T21:46:41.945ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "7d53aaa2-9938-444f-8195-7684ff0866e5", "date": "Tue, 26 Aug 2025 21:46:41 GMT"} })
13672025-08-26T21:46:41.946ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 10.260018541s iteration = 2 status = {}
13682025-08-26T21:46:46.665ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 10 reason = Timeout
13692025-08-26T21:46:46.665ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
13702025-08-26T21:46:46.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
13712025-08-26T21:46:46.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13722025-08-26T21:46:46.670ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
13732025-08-26T21:46:46.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
13742025-08-26T21:46:46.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
13752025-08-26T21:46:46.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
13762025-08-26T21:46:46.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.503449ms iteration = 10 status = {"chicken_switches_updated":false}
13772025-08-26T21:46:51.665ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 11 reason = Timeout
13782025-08-26T21:46:51.665ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
13792025-08-26T21:46:51.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
13802025-08-26T21:46:51.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13812025-08-26T21:46:51.670ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
13822025-08-26T21:46:51.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
13832025-08-26T21:46:51.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
13842025-08-26T21:46:51.672ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
13852025-08-26T21:46:51.672ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.269041ms iteration = 11 status = {"chicken_switches_updated":false}
13862025-08-26T21:46:51.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 6 reason = Timeout
13872025-08-26T21:46:51.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13882025-08-26T21:46:51.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13892025-08-26T21:46:51.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13902025-08-26T21:46:51.678ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13912025-08-26T21:46:51.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13922025-08-26T21:46:51.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13932025-08-26T21:46:51.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13942025-08-26T21:46:51.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13952025-08-26T21:46:51.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13962025-08-26T21:46:51.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
13972025-08-26T21:46:51.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13982025-08-26T21:46:51.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
13992025-08-26T21:46:51.685ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 6 reason = Timeout
14002025-08-26T21:46:51.685ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14012025-08-26T21:46:51.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14022025-08-26T21:46:51.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14032025-08-26T21:46:51.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14042025-08-26T21:46:51.700ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
14052025-08-26T21:46:51.700ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.270762ms iteration = 6 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
14062025-08-26T21:46:51.721ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.533441ms iteration = 6 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
14072025-08-26T21:46:56.668ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 12 reason = Timeout
14082025-08-26T21:46:56.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14092025-08-26T21:46:56.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
14102025-08-26T21:46:56.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14112025-08-26T21:46:56.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
14122025-08-26T21:46:56.673ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14132025-08-26T21:46:56.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
14142025-08-26T21:46:56.675ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
14152025-08-26T21:46:56.675ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.370466ms iteration = 12 status = {"chicken_switches_updated":false}
14162025-08-26T21:47:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 3 reason = Timeout
14172025-08-26T21:47:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 2 reason = Timeout
14182025-08-26T21:47:01.665ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
14192025-08-26T21:47:01.665ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
14202025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 19.87\u{b5}s iteration = 3 status = {"error":"no config"}
14212025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14222025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 2 reason = Timeout
14232025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 2 reason = Timeout
14242025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14252025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 2 reason = Timeout
14262025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 3 reason = Timeout
14272025-08-26T21:47:01.666ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
14282025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 2 reason = Timeout
14292025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 12.54\u{b5}s iteration = 3 status = {"error":"no config"}
14302025-08-26T21:47:01.666ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:37:01.665973227 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
14312025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
14322025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 3 reason = Timeout
14332025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
14342025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 3 reason = Timeout
14352025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
14362025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 2 reason = Timeout
14372025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
14382025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14392025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 3 reason = Timeout
14402025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14412025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14422025-08-26T21:47:01.666ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 925, tv_nsec: 523489624 } })
14432025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
14442025-08-26T21:47:01.666ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 925, tv_nsec: 523503774 } })
14452025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
14462025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
14472025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14482025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 3 reason = Timeout
14492025-08-26T21:47:01.666ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
14502025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 15.2\u{b5}s iteration = 3 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
14512025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
14522025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 928.171\u{b5}s iteration = 2 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
14532025-08-26T21:47:01.666ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
14542025-08-26T21:47:01.667ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
14552025-08-26T21:47:01.667ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.108425ms iteration = 2 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
14562025-08-26T21:47:01.667ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
14572025-08-26T21:47:01.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
14582025-08-26T21:47:01.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
14592025-08-26T21:47:01.668ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14602025-08-26T21:47:01.668ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 13 reason = Timeout
14612025-08-26T21:47:01.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14622025-08-26T21:47:01.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
14632025-08-26T21:47:01.669ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14642025-08-26T21:47:01.669ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 925, tv_nsec: 526288457 } })
14652025-08-26T21:47:01.669ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
14662025-08-26T21:47:01.669ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:37:01.665973227 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
14672025-08-26T21:47:01.669ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.57201ms iteration = 2 status = {"expiration":"2025-08-26T21:37:01.665973227Z","pruned":[]}
14682025-08-26T21:47:01.669ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.622762ms iteration = 3 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
14692025-08-26T21:47:01.669ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14702025-08-26T21:47:01.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14712025-08-26T21:47:01.671ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 925, tv_nsec: 528626590 } })
14722025-08-26T21:47:01.671ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
14732025-08-26T21:47:01.671ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14742025-08-26T21:47:01.671ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
14752025-08-26T21:47:01.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14762025-08-26T21:47:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
14772025-08-26T21:47:01.674ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
14782025-08-26T21:47:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
14792025-08-26T21:47:01.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 4 reason = Timeout
14802025-08-26T21:47:01.674ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
14812025-08-26T21:47:01.674ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 9.56\u{b5}s iteration = 4 status = "Disabled"
14822025-08-26T21:47:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
14832025-08-26T21:47:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
14842025-08-26T21:47:01.675ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14852025-08-26T21:47:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
14862025-08-26T21:47:01.675ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 2 reason = Timeout
14872025-08-26T21:47:01.675ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
14882025-08-26T21:47:01.675ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
14892025-08-26T21:47:01.675ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 3 reason = Timeout
14902025-08-26T21:47:01.675ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14912025-08-26T21:47:01.675ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 3 reason = Timeout
14922025-08-26T21:47:01.675ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 4 reason = Timeout
14932025-08-26T21:47:01.675ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
14942025-08-26T21:47:01.675ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14952025-08-26T21:47:01.675ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 3 reason = Timeout
14962025-08-26T21:47:01.675ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14972025-08-26T21:47:01.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 7 reason = Timeout
14982025-08-26T21:47:01.675ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 3 reason = Timeout
14992025-08-26T21:47:01.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15002025-08-26T21:47:01.675ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
15012025-08-26T21:47:01.675ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 3 reason = Timeout
15022025-08-26T21:47:01.675ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15032025-08-26T21:47:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
15042025-08-26T21:47:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
15052025-08-26T21:47:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
15062025-08-26T21:47:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15072025-08-26T21:47:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
15082025-08-26T21:47:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15092025-08-26T21:47:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15102025-08-26T21:47:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15112025-08-26T21:47:01.676ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15122025-08-26T21:47:01.677ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
15132025-08-26T21:47:01.677ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
15142025-08-26T21:47:01.677ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
15152025-08-26T21:47:01.677ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15162025-08-26T21:47:01.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15172025-08-26T21:47:01.678ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
15182025-08-26T21:47:01.678ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 9.687399ms iteration = 13 status = {"chicken_switches_updated":false}
15192025-08-26T21:47:01.678ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15202025-08-26T21:47:01.678ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15212025-08-26T21:47:01.678ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15222025-08-26T21:47:01.678ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15232025-08-26T21:47:01.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15242025-08-26T21:47:01.679ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15252025-08-26T21:47:01.679ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15262025-08-26T21:47:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
15272025-08-26T21:47:01.683ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
15282025-08-26T21:47:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
15292025-08-26T21:47:01.684ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
15302025-08-26T21:47:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
15312025-08-26T21:47:01.684ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15322025-08-26T21:47:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
15332025-08-26T21:47:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
15342025-08-26T21:47:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
15352025-08-26T21:47:01.684ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
15362025-08-26T21:47:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15372025-08-26T21:47:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
15382025-08-26T21:47:01.685ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15392025-08-26T21:47:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15402025-08-26T21:47:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
15412025-08-26T21:47:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
15422025-08-26T21:47:01.687ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15432025-08-26T21:47:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15442025-08-26T21:47:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
15452025-08-26T21:47:01.687ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
15462025-08-26T21:47:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
15472025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
15482025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15492025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15502025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 2 reason = Timeout
15512025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 3 reason = Timeout
15522025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
15532025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
15542025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
15552025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 7 reason = Timeout
15562025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15572025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 3 reason = Timeout
15582025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
15592025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 2 reason = Timeout
15602025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
15612025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 3 reason = Timeout
15622025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
15632025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 2 reason = Timeout
15642025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
15652025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 2 reason = Timeout
15662025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
15672025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
15682025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 3 reason = Timeout
15692025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 10.9\u{b5}s iteration = 3 status = {"disabled":true,"errors":[],"sps":[]}
15702025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 2 reason = Timeout
15712025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
15722025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 3 reason = Timeout
15732025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 3 reason = Timeout
15742025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
15752025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
15762025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 3 reason = Timeout
15772025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
15782025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 3 reason = Timeout
15792025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
15802025-08-26T21:47:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15812025-08-26T21:47:01.689ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 3 reason = Timeout
15822025-08-26T21:47:01.689ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
15832025-08-26T21:47:01.689ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15842025-08-26T21:47:01.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
15852025-08-26T21:47:01.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
15862025-08-26T21:47:01.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
15872025-08-26T21:47:01.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
15882025-08-26T21:47:01.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
15892025-08-26T21:47:01.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
15902025-08-26T21:47:01.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
15912025-08-26T21:47:01.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
15922025-08-26T21:47:01.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
15932025-08-26T21:47:01.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
15942025-08-26T21:47:01.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
15952025-08-26T21:47:01.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15962025-08-26T21:47:01.695ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15972025-08-26T21:47:01.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
15982025-08-26T21:47:01.695ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15992025-08-26T21:47:01.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16002025-08-26T21:47:01.696ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 925, tv_nsec: 553448609 } })
16012025-08-26T21:47:01.696ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
16022025-08-26T21:47:01.697ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
16032025-08-26T21:47:01.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
16042025-08-26T21:47:01.699ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 10.67398ms iteration = 2 status = {"errors":[],"found_port_ok":[]}
16052025-08-26T21:47:01.702ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 13.510724ms iteration = 3 status = {"errors":[],"requests_created_ok":[]}
16062025-08-26T21:47:01.705ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16072025-08-26T21:47:01.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16082025-08-26T21:47:01.706ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
16092025-08-26T21:47:01.706ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 17.77139ms iteration = 2 status = {"by_rx":{},"error":null}
16102025-08-26T21:47:01.711ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
16112025-08-26T21:47:01.711ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16122025-08-26T21:47:01.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
16132025-08-26T21:47:01.714ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16142025-08-26T21:47:01.718ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
16152025-08-26T21:47:01.718ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16162025-08-26T21:47:01.718ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
16172025-08-26T21:47:01.718ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16182025-08-26T21:47:01.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16192025-08-26T21:47:01.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
16202025-08-26T21:47:01.768ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
16212025-08-26T21:47:01.768ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
16222025-08-26T21:47:01.768ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
16232025-08-26T21:47:01.768ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16242025-08-26T21:47:01.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16252025-08-26T21:47:01.771ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
16262025-08-26T21:47:01.771ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
16272025-08-26T21:47:01.772ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3079442-1bb8-4141-bf10-e333dfa8a0c6", "content-length": "26", "date": "Tue, 26 Aug 2025 21:47:01 GMT"} })
16282025-08-26T21:47:01.772ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
16292025-08-26T21:47:01.773ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 84.646268ms iteration = 2 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
16302025-08-26T21:47:01.774ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
16312025-08-26T21:47:01.785ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16322025-08-26T21:47:01.788ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
16332025-08-26T21:47:01.788ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
16342025-08-26T21:47:01.788ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
16352025-08-26T21:47:01.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
16362025-08-26T21:47:01.789ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
16372025-08-26T21:47:01.789ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
16382025-08-26T21:47:01.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
16392025-08-26T21:47:01.790ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16402025-08-26T21:47:01.792ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
16412025-08-26T21:47:01.792ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
16422025-08-26T21:47:01.792ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
16432025-08-26T21:47:01.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
16442025-08-26T21:47:01.794ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
16452025-08-26T21:47:01.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
16462025-08-26T21:47:01.794ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16472025-08-26T21:47:01.794ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 106.02919ms iteration = 3 status = {"errors":[],"finish_invoked_ok":[]}
16482025-08-26T21:47:01.795ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
16492025-08-26T21:47:01.795ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
16502025-08-26T21:47:01.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
16512025-08-26T21:47:01.797ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
16522025-08-26T21:47:01.797ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16532025-08-26T21:47:01.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16542025-08-26T21:47:01.801ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
16552025-08-26T21:47:01.801ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
16562025-08-26T21:47:01.801ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 113.10353ms iteration = 2 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
16572025-08-26T21:47:01.802ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16582025-08-26T21:47:01.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16592025-08-26T21:47:01.803ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16602025-08-26T21:47:01.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16612025-08-26T21:47:01.804ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 115.635087ms iteration = 3 status = {"errors":[],"garbage_collect_requested":[]}
16622025-08-26T21:47:01.804ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16632025-08-26T21:47:01.804ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 116.25619ms iteration = 7 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
16642025-08-26T21:47:01.806ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
16652025-08-26T21:47:01.806ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 117.607741ms iteration = 3 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
16662025-08-26T21:47:01.808ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
16672025-08-26T21:47:01.808ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16682025-08-26T21:47:01.808ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
16692025-08-26T21:47:01.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16702025-08-26T21:47:01.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
16712025-08-26T21:47:01.811ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
16722025-08-26T21:47:01.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
16732025-08-26T21:47:01.816ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
16742025-08-26T21:47:01.816ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
16752025-08-26T21:47:01.816ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 140.798244ms iteration = 2 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
16762025-08-26T21:47:01.816ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
16772025-08-26T21:47:01.816ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
16782025-08-26T21:47:01.817ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d73fced2-c7b0-4138-a5f7-e43871cdb2c9", "content-length": "26", "date": "Tue, 26 Aug 2025 21:47:01 GMT"} })
16792025-08-26T21:47:01.817ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
16802025-08-26T21:47:01.819ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
16812025-08-26T21:47:01.825ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16822025-08-26T21:47:01.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16832025-08-26T21:47:01.827ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16842025-08-26T21:47:01.831ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
16852025-08-26T21:47:01.831ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16862025-08-26T21:47:01.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16872025-08-26T21:47:01.837ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16882025-08-26T21:47:01.840ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
16892025-08-26T21:47:01.840ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16902025-08-26T21:47:01.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
16912025-08-26T21:47:01.842ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
16922025-08-26T21:47:01.842ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
16932025-08-26T21:47:01.843ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab101acc-6fef-47f5-a30a-98f403f572db", "content-length": "26", "date": "Tue, 26 Aug 2025 21:47:01 GMT"} })
16942025-08-26T21:47:01.843ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
16952025-08-26T21:47:01.844ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
16962025-08-26T21:47:01.844ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
16972025-08-26T21:47:01.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
16982025-08-26T21:47:01.847ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 158.155705ms iteration = 3 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
16992025-08-26T21:47:01.860ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 184.52561ms iteration = 4 status = {"physical_disks_added":0}
17002025-08-26T21:47:01.864ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 188.602712ms iteration = 3 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
17012025-08-26T21:47:01.864ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
17022025-08-26T21:47:01.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
17032025-08-26T21:47:01.869ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17042025-08-26T21:47:01.871ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 205.077583ms iteration = 3 status = {}
17052025-08-26T21:47:01.872ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
17062025-08-26T21:47:01.872ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
17072025-08-26T21:47:01.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
17082025-08-26T21:47:01.880ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 191.668321ms iteration = 3 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
17092025-08-26T21:47:01.892ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
17102025-08-26T21:47:01.892ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
17112025-08-26T21:47:01.892ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 203.843035ms iteration = 3 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
17122025-08-26T21:47:01.903ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17132025-08-26T21:47:01.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17142025-08-26T21:47:01.906ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17152025-08-26T21:47:01.908ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
17162025-08-26T21:47:01.908ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17172025-08-26T21:47:01.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17182025-08-26T21:47:01.913ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
17192025-08-26T21:47:01.913ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 238.179349ms iteration = 3 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
17202025-08-26T21:47:01.916ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
17212025-08-26T21:47:01.916ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
17222025-08-26T21:47:01.916ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
17232025-08-26T21:47:01.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17242025-08-26T21:47:01.918ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17252025-08-26T21:47:01.918ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
17262025-08-26T21:47:01.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
17272025-08-26T21:47:01.920ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 254.65469ms iteration = 3 status = null
17282025-08-26T21:47:01.924ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
17292025-08-26T21:47:01.924ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4a4e022-11a9-4b27-af64-d637789ab0a8", "content-length": "691", "date": "Tue, 26 Aug 2025 21:47:01 GMT"} })
17302025-08-26T21:47:01.925ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
17312025-08-26T21:47:01.925ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
17322025-08-26T21:47:01.925ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
17332025-08-26T21:47:01.925ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
17342025-08-26T21:47:01.926ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
17352025-08-26T21:47:01.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17362025-08-26T21:47:01.927ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 251.613112ms iteration = 3 status = {}
17372025-08-26T21:47:01.931ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
17382025-08-26T21:47:01.931ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 255.861767ms iteration = 7 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
17392025-08-26T21:47:01.935ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
17402025-08-26T21:47:01.935ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
17412025-08-26T21:47:01.935ZINFOnexus (BackgroundTasks): found latest generation (first find) background_task = dns_config_internal dns_group = internal file = nexus/src/app/background/tasks/dns_config.rs:87 generation = 1
17422025-08-26T21:47:01.935ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 269.455974ms iteration = 2 status = {"generation":1}
17432025-08-26T21:47:01.935ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 4 reason = Dependency
17442025-08-26T21:47:01.970ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
17452025-08-26T21:47:01.970ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 0 version = 2
17462025-08-26T21:47:01.970ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17472025-08-26T21:47:01.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
17482025-08-26T21:47:01.973ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17492025-08-26T21:47:01.976ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17502025-08-26T21:47:01.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17512025-08-26T21:47:01.978ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
17522025-08-26T21:47:01.978ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17532025-08-26T21:47:01.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
17542025-08-26T21:47:01.979ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17552025-08-26T21:47:01.979ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17562025-08-26T21:47:01.980ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 4 version = 2
17572025-08-26T21:47:01.980ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
17582025-08-26T21:47:01.980ZINFOnexus (BackgroundTasks): found latest generation (first find) background_task = dns_config_external dns_group = external file = nexus/src/app/background/tasks/dns_config.rs:87 generation = 2
17592025-08-26T21:47:01.980ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 315.006471ms iteration = 2 status = {"generation":2}
17602025-08-26T21:47:01.980ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 4 reason = Dependency
17612025-08-26T21:47:01.980ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17622025-08-26T21:47:01.983ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
17632025-08-26T21:47:01.983ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
17642025-08-26T21:47:01.984ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
17652025-08-26T21:47:01.985ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (initial) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-08-26T21:45:59.866217Z, time_modified: 2025-08-26T21:45:59.866217Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:77
17662025-08-26T21:47:01.985ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 319.233816ms iteration = 2 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]}},"default_endpoint":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
17672025-08-26T21:47:01.986ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
17682025-08-26T21:47:01.986ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 297.535317ms iteration = 2 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
17692025-08-26T21:47:01.986ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 310.658514ms iteration = 3 status = 0
17702025-08-26T21:47:02.031ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
17712025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = / versions = all
17722025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /access versions = all
17732025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /assets/{path} versions = all
17742025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /device/auth versions = all
17752025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /device/confirm versions = all
17762025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /device/success versions = all
17772025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /device/token versions = all
17782025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /device/verify versions = all
17792025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/probes versions = all
17802025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /experimental/v1/probes versions = all
17812025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /experimental/v1/probes/{probe} versions = all
17822025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/probes/{probe} versions = all
17832025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/system/support-bundles versions = all
17842025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /experimental/v1/system/support-bundles versions = all
17852025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
17862025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
17872025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
17882025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
17892025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
17902025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
17912025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
17922025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/index versions = all
17932025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /images versions = all
17942025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /login versions = all
17952025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /login/{silo_name}/local versions = all
17962025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
17972025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
17982025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
17992025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /lookup/{path} versions = all
18002025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /projects/{path} versions = all
18012025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /projects-new versions = all
18022025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /settings/{path} versions = all
18032025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /system/{path} versions = all
18042025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /utilization versions = all
18052025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/affinity-groups versions = all
18062025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/affinity-groups versions = all
18072025-08-26T21:47:02.039ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/affinity-groups/{affinity_group} versions = all
18082025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/affinity-groups/{affinity_group} versions = all
18092025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/affinity-groups/{affinity_group} versions = all
18102025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/affinity-groups/{affinity_group}/members versions = all
18112025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
18122025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
18132025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
18142025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/alert-classes versions = all
18152025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/alert-receivers versions = all
18162025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/alert-receivers/{receiver} versions = all
18172025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/alert-receivers/{receiver} versions = all
18182025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/alert-receivers/{receiver}/deliveries versions = all
18192025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/alert-receivers/{receiver}/probe versions = all
18202025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/alert-receivers/{receiver}/subscriptions versions = all
18212025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/alert-receivers/{receiver}/subscriptions/{subscription} versions = all
18222025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/alerts/{alert_id}/resend versions = all
18232025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/anti-affinity-groups versions = all
18242025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/anti-affinity-groups versions = all
18252025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
18262025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
18272025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
18282025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members versions = all
18292025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
18302025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
18312025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
18322025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/auth-settings versions = all
18332025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/auth-settings versions = all
18342025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/certificates versions = all
18352025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/certificates versions = all
18362025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/certificates/{certificate} versions = all
18372025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/certificates/{certificate} versions = all
18382025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/disks versions = all
18392025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/disks versions = all
18402025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/disks/{disk} versions = all
18412025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/disks/{disk} versions = all
18422025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/disks/{disk}/bulk-write versions = all
18432025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
18442025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
18452025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/disks/{disk}/finalize versions = all
18462025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/floating-ips versions = all
18472025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/floating-ips versions = all
18482025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
18492025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/floating-ips/{floating_ip} versions = all
18502025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
18512025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
18522025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
18532025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/groups versions = all
18542025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/groups/{group_id} versions = all
18552025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/images versions = all
18562025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/images versions = all
18572025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/images/{image} versions = all
18582025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/images/{image} versions = all
18592025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/images/{image}/demote versions = all
18602025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/images/{image}/promote versions = all
18612025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances versions = all
18622025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances versions = all
18632025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/instances/{instance} versions = all
18642025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance} versions = all
18652025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/instances/{instance} versions = all
18662025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/affinity-groups versions = all
18672025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/anti-affinity-groups versions = all
18682025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/disks versions = all
18692025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances/{instance}/disks/attach versions = all
18702025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances/{instance}/disks/detach versions = all
18712025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/external-ips versions = all
18722025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
18732025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
18742025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances/{instance}/reboot versions = all
18752025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/serial-console versions = all
18762025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
18772025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
18782025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances/{instance}/start versions = all
18792025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances/{instance}/stop versions = all
18802025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/internet-gateway-ip-addresses versions = all
18812025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/internet-gateway-ip-addresses versions = all
18822025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
18832025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/internet-gateway-ip-pools versions = all
18842025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/internet-gateway-ip-pools versions = all
18852025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
18862025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/internet-gateways versions = all
18872025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/internet-gateways versions = all
18882025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
18892025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/internet-gateways/{gateway} versions = all
18902025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/ip-pools versions = all
18912025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/ip-pools/{pool} versions = all
18922025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/login/{silo_name}/local versions = all
18932025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/logout versions = all
18942025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/me versions = all
18952025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/me/access-tokens versions = all
18962025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/me/access-tokens/{token_id} versions = all
18972025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/me/groups versions = all
18982025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/me/ssh-keys versions = all
18992025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/me/ssh-keys versions = all
19002025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
19012025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
19022025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/metrics/{metric_name} versions = all
19032025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/network-interfaces versions = all
19042025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/network-interfaces versions = all
19052025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/network-interfaces/{interface} versions = all
19062025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/network-interfaces/{interface} versions = all
19072025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/network-interfaces/{interface} versions = all
19082025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/ping versions = all
19092025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/policy versions = all
19102025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/policy versions = all
19112025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/projects versions = all
19122025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/projects versions = all
19132025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/projects/{project} versions = all
19142025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/projects/{project} versions = all
19152025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/projects/{project} versions = all
19162025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/projects/{project}/policy versions = all
19172025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/projects/{project}/policy versions = all
19182025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/snapshots versions = all
19192025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/snapshots versions = all
19202025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/snapshots/{snapshot} versions = all
19212025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/snapshots/{snapshot} versions = all
19222025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/audit-log versions = all
19232025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/disks versions = all
19242025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
19252025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/rack-switch-port/{rack_id}/{switch_location}/{port}/lldp/neighbors versions = all
19262025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/racks versions = all
19272025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
19282025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/sleds versions = all
19292025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/hardware/sleds versions = all
19302025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
19312025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
19322025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
19332025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
19342025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
19352025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/switch-port versions = all
19362025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
19372025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
19382025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
19392025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
19402025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
19412025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/switches versions = all
19422025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
19432025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/identity-providers versions = all
19442025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/identity-providers/local/users versions = all
19452025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
19462025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
19472025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/identity-providers/saml versions = all
19482025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
19492025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools versions = all
19502025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/ip-pools versions = all
19512025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
19522025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools/{pool} versions = all
19532025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/ip-pools/{pool} versions = all
19542025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
19552025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
19562025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
19572025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
19582025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
19592025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
19602025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
19612025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
19622025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools-service versions = all
19632025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools-service/ranges versions = all
19642025-08-26T21:47:02.040ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
19652025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
19662025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/metrics/{metric_name} versions = all
19672025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/address-lot versions = all
19682025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/networking/address-lot versions = all
19692025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
19702025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/address-lot/{address_lot} versions = all
19712025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
19722025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/allow-list versions = all
19732025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/networking/allow-list versions = all
19742025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/networking/bfd-disable versions = all
19752025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/networking/bfd-enable versions = all
19762025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bfd-status versions = all
19772025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/networking/bgp versions = all
19782025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp versions = all
19792025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/networking/bgp versions = all
19802025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp-announce-set versions = all
19812025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
19822025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
19832025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
19842025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp-exported versions = all
19852025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp-message-history versions = all
19862025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
19872025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp-status versions = all
19882025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/inbound-icmp versions = all
19892025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/networking/inbound-icmp versions = all
19902025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/loopback-address versions = all
19912025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/networking/loopback-address versions = all
19922025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
19932025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
19942025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/switch-port-settings versions = all
19952025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/networking/switch-port-settings versions = all
19962025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
19972025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/policy versions = all
19982025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/policy versions = all
19992025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/silo-quotas versions = all
20002025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/silos versions = all
20012025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/silos versions = all
20022025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/silos/{silo} versions = all
20032025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/silos/{silo} versions = all
20042025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
20052025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/silos/{silo}/policy versions = all
20062025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/silos/{silo}/policy versions = all
20072025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/silos/{silo}/quotas versions = all
20082025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
20092025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/timeseries/query versions = all
20102025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/timeseries/schemas versions = all
20112025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/update/repository versions = all
20122025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/update/repository/{system_version} versions = all
20132025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/update/target-release versions = all
20142025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/update/target-release versions = all
20152025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/update/trust-roots versions = all
20162025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/update/trust-roots versions = all
20172025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/update/trust-roots/{trust_root_id} versions = all
20182025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/update/trust-roots/{trust_root_id} versions = all
20192025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/users versions = all
20202025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/users/{user_id} versions = all
20212025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/users-builtin versions = all
20222025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/users-builtin/{user} versions = all
20232025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/utilization/silos versions = all
20242025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/utilization/silos/{silo} versions = all
20252025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/timeseries/query versions = all
20262025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/users versions = all
20272025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/users/{user_id} versions = all
20282025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/users/{user_id}/access-tokens versions = all
20292025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/users/{user_id}/logout versions = all
20302025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/users/{user_id}/sessions versions = all
20312025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/utilization versions = all
20322025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-firewall-rules versions = all
20332025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/vpc-firewall-rules versions = all
20342025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-router-routes versions = all
20352025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/vpc-router-routes versions = all
20362025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
20372025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-router-routes/{route} versions = all
20382025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/vpc-router-routes/{route} versions = all
20392025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-routers versions = all
20402025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/vpc-routers versions = all
20412025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/vpc-routers/{router} versions = all
20422025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-routers/{router} versions = all
20432025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/vpc-routers/{router} versions = all
20442025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-subnets versions = all
20452025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/vpc-subnets versions = all
20462025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
20472025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-subnets/{subnet} versions = all
20482025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
20492025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
20502025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpcs versions = all
20512025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/vpcs versions = all
20522025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/vpcs/{vpc} versions = all
20532025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpcs/{vpc} versions = all
20542025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/vpcs/{vpc} versions = all
20552025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/webhook-receivers versions = all
20562025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/webhook-receivers/{receiver} versions = all
20572025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/webhook-secrets versions = all
20582025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/webhook-secrets versions = all
20592025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/webhook-secrets/{secret_id} versions = all
20602025-08-26T21:47:02.041ZINFOnexus (dropshot_external): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:328 local_addr = 172.30.2.5:443
20612025-08-26T21:47:02.041ZDEBGnexus (dropshot_external): successfully registered DTrace USDT probes local_addr = 172.30.2.5:443
20622025-08-26T21:47:02.049ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = / versions = all
20632025-08-26T21:47:02.049ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /access versions = all
20642025-08-26T21:47:02.049ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /assets/{path} versions = all
20652025-08-26T21:47:02.049ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /device/auth versions = all
20662025-08-26T21:47:02.049ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /device/confirm versions = all
20672025-08-26T21:47:02.049ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /device/success versions = all
20682025-08-26T21:47:02.049ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /device/token versions = all
20692025-08-26T21:47:02.049ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /device/verify versions = all
20702025-08-26T21:47:02.049ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/probes versions = all
20712025-08-26T21:47:02.049ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /experimental/v1/probes versions = all
20722025-08-26T21:47:02.049ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /experimental/v1/probes/{probe} versions = all
20732025-08-26T21:47:02.049ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/probes/{probe} versions = all
20742025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/system/support-bundles versions = all
20752025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /experimental/v1/system/support-bundles versions = all
20762025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
20772025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
20782025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
20792025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
20802025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
20812025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
20822025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
20832025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/index versions = all
20842025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /images versions = all
20852025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /login versions = all
20862025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /login/{silo_name}/local versions = all
20872025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
20882025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
20892025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
20902025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /lookup/{path} versions = all
20912025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /projects/{path} versions = all
20922025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /projects-new versions = all
20932025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /settings/{path} versions = all
20942025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /system/{path} versions = all
20952025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /utilization versions = all
20962025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/affinity-groups versions = all
20972025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/affinity-groups versions = all
20982025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/affinity-groups/{affinity_group} versions = all
20992025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/affinity-groups/{affinity_group} versions = all
21002025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/affinity-groups/{affinity_group} versions = all
21012025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/affinity-groups/{affinity_group}/members versions = all
21022025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
21032025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
21042025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
21052025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/alert-classes versions = all
21062025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/alert-receivers versions = all
21072025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/alert-receivers/{receiver} versions = all
21082025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/alert-receivers/{receiver} versions = all
21092025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/alert-receivers/{receiver}/deliveries versions = all
21102025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/alert-receivers/{receiver}/probe versions = all
21112025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/alert-receivers/{receiver}/subscriptions versions = all
21122025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/alert-receivers/{receiver}/subscriptions/{subscription} versions = all
21132025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/alerts/{alert_id}/resend versions = all
21142025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/anti-affinity-groups versions = all
21152025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/anti-affinity-groups versions = all
21162025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
21172025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
21182025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
21192025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members versions = all
21202025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
21212025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
21222025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
21232025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/auth-settings versions = all
21242025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/auth-settings versions = all
21252025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/certificates versions = all
21262025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/certificates versions = all
21272025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/certificates/{certificate} versions = all
21282025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/certificates/{certificate} versions = all
21292025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/disks versions = all
21302025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/disks versions = all
21312025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/disks/{disk} versions = all
21322025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/disks/{disk} versions = all
21332025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/disks/{disk}/bulk-write versions = all
21342025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
21352025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
21362025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/disks/{disk}/finalize versions = all
21372025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/floating-ips versions = all
21382025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/floating-ips versions = all
21392025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
21402025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/floating-ips/{floating_ip} versions = all
21412025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
21422025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
21432025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
21442025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/groups versions = all
21452025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/groups/{group_id} versions = all
21462025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/images versions = all
21472025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/images versions = all
21482025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/images/{image} versions = all
21492025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/images/{image} versions = all
21502025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/images/{image}/demote versions = all
21512025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/images/{image}/promote versions = all
21522025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances versions = all
21532025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances versions = all
21542025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/instances/{instance} versions = all
21552025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance} versions = all
21562025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/instances/{instance} versions = all
21572025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/affinity-groups versions = all
21582025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/anti-affinity-groups versions = all
21592025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/disks versions = all
21602025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances/{instance}/disks/attach versions = all
21612025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances/{instance}/disks/detach versions = all
21622025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/external-ips versions = all
21632025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
21642025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
21652025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances/{instance}/reboot versions = all
21662025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/serial-console versions = all
21672025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
21682025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
21692025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances/{instance}/start versions = all
21702025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances/{instance}/stop versions = all
21712025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/internet-gateway-ip-addresses versions = all
21722025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/internet-gateway-ip-addresses versions = all
21732025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
21742025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/internet-gateway-ip-pools versions = all
21752025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/internet-gateway-ip-pools versions = all
21762025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
21772025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/internet-gateways versions = all
21782025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/internet-gateways versions = all
21792025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
21802025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/internet-gateways/{gateway} versions = all
21812025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/ip-pools versions = all
21822025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/ip-pools/{pool} versions = all
21832025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/login/{silo_name}/local versions = all
21842025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/logout versions = all
21852025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/me versions = all
21862025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/me/access-tokens versions = all
21872025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/me/access-tokens/{token_id} versions = all
21882025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/me/groups versions = all
21892025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/me/ssh-keys versions = all
21902025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/me/ssh-keys versions = all
21912025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
21922025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
21932025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/metrics/{metric_name} versions = all
21942025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/network-interfaces versions = all
21952025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/network-interfaces versions = all
21962025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/network-interfaces/{interface} versions = all
21972025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/network-interfaces/{interface} versions = all
21982025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/network-interfaces/{interface} versions = all
21992025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/ping versions = all
22002025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/policy versions = all
22012025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/policy versions = all
22022025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/projects versions = all
22032025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/projects versions = all
22042025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/projects/{project} versions = all
22052025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/projects/{project} versions = all
22062025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/projects/{project} versions = all
22072025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/projects/{project}/policy versions = all
22082025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/projects/{project}/policy versions = all
22092025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/snapshots versions = all
22102025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/snapshots versions = all
22112025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/snapshots/{snapshot} versions = all
22122025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/snapshots/{snapshot} versions = all
22132025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/audit-log versions = all
22142025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/disks versions = all
22152025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
22162025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/rack-switch-port/{rack_id}/{switch_location}/{port}/lldp/neighbors versions = all
22172025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/racks versions = all
22182025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
22192025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/sleds versions = all
22202025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/hardware/sleds versions = all
22212025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
22222025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
22232025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
22242025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
22252025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
22262025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/switch-port versions = all
22272025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
22282025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
22292025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
22302025-08-26T21:47:02.050ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
22312025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
22322025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/switches versions = all
22332025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
22342025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/identity-providers versions = all
22352025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/identity-providers/local/users versions = all
22362025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
22372025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
22382025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/identity-providers/saml versions = all
22392025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
22402025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools versions = all
22412025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/ip-pools versions = all
22422025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
22432025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools/{pool} versions = all
22442025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/ip-pools/{pool} versions = all
22452025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
22462025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
22472025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
22482025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
22492025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
22502025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
22512025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
22522025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
22532025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools-service versions = all
22542025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools-service/ranges versions = all
22552025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
22562025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
22572025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/metrics/{metric_name} versions = all
22582025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/address-lot versions = all
22592025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/networking/address-lot versions = all
22602025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
22612025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/address-lot/{address_lot} versions = all
22622025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
22632025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/allow-list versions = all
22642025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/networking/allow-list versions = all
22652025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/networking/bfd-disable versions = all
22662025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/networking/bfd-enable versions = all
22672025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bfd-status versions = all
22682025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/networking/bgp versions = all
22692025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp versions = all
22702025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/networking/bgp versions = all
22712025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp-announce-set versions = all
22722025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
22732025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
22742025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
22752025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp-exported versions = all
22762025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp-message-history versions = all
22772025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
22782025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp-status versions = all
22792025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/inbound-icmp versions = all
22802025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/networking/inbound-icmp versions = all
22812025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/loopback-address versions = all
22822025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/networking/loopback-address versions = all
22832025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
22842025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
22852025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/switch-port-settings versions = all
22862025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/networking/switch-port-settings versions = all
22872025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
22882025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/policy versions = all
22892025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/policy versions = all
22902025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/silo-quotas versions = all
22912025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/silos versions = all
22922025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/silos versions = all
22932025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/silos/{silo} versions = all
22942025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/silos/{silo} versions = all
22952025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
22962025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/silos/{silo}/policy versions = all
22972025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/silos/{silo}/policy versions = all
22982025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/silos/{silo}/quotas versions = all
22992025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
23002025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/timeseries/query versions = all
23012025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/timeseries/schemas versions = all
23022025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/update/repository versions = all
23032025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/update/repository/{system_version} versions = all
23042025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/update/target-release versions = all
23052025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/update/target-release versions = all
23062025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/update/trust-roots versions = all
23072025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/update/trust-roots versions = all
23082025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/update/trust-roots/{trust_root_id} versions = all
23092025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/update/trust-roots/{trust_root_id} versions = all
23102025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/users versions = all
23112025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/users/{user_id} versions = all
23122025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/users-builtin versions = all
23132025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/users-builtin/{user} versions = all
23142025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/utilization/silos versions = all
23152025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/utilization/silos/{silo} versions = all
23162025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/timeseries/query versions = all
23172025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/users versions = all
23182025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/users/{user_id} versions = all
23192025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/users/{user_id}/access-tokens versions = all
23202025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/users/{user_id}/logout versions = all
23212025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/users/{user_id}/sessions versions = all
23222025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/utilization versions = all
23232025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-firewall-rules versions = all
23242025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/vpc-firewall-rules versions = all
23252025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-router-routes versions = all
23262025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/vpc-router-routes versions = all
23272025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
23282025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-router-routes/{route} versions = all
23292025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/vpc-router-routes/{route} versions = all
23302025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-routers versions = all
23312025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/vpc-routers versions = all
23322025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/vpc-routers/{router} versions = all
23332025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-routers/{router} versions = all
23342025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/vpc-routers/{router} versions = all
23352025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-subnets versions = all
23362025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/vpc-subnets versions = all
23372025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
23382025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-subnets/{subnet} versions = all
23392025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
23402025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
23412025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpcs versions = all
23422025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/vpcs versions = all
23432025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/vpcs/{vpc} versions = all
23442025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpcs/{vpc} versions = all
23452025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/vpcs/{vpc} versions = all
23462025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/webhook-receivers versions = all
23472025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/webhook-receivers/{receiver} versions = all
23482025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/webhook-secrets versions = all
23492025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/webhook-secrets versions = all
23502025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/webhook-secrets/{secret_id} versions = all
23512025-08-26T21:47:02.051ZINFOnexus (dropshot_external_techport): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:328 local_addr = [fd00:1122:3344:101::a]:12228
23522025-08-26T21:47:02.051ZDEBGnexus (dropshot_external_techport): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::a]:12228
23532025-08-26T21:47:02.052ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
23542025-08-26T21:47:02.052ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
23552025-08-26T21:47:02.053ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "fbf8e019-9a98-42c8-9ec4-224017986174", "content-length": "372", "date": "Tue, 26 Aug 2025 21:47:01 GMT"} })
23562025-08-26T21:47:02.053ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
23572025-08-26T21:47:02.053ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
23582025-08-26T21:47:02.058ZDEBGnexus (nexus-producer-server): registered DTrace probes
23592025-08-26T21:47:02.066ZDEBGnexus (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::a]:35154 method = GET path = /{producer_id} versions = all
23602025-08-26T21:47:02.067ZINFOnexus (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:328 local_addr = [fd00:1122:3344:101::a]:35154
23612025-08-26T21:47:02.067ZDEBGnexus (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::a]:35154
23622025-08-26T21:47:02.067ZDEBGnexus (nexus-producer-server): Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::a]:35154
23632025-08-26T21:47:02.067ZDEBGnexus (nexus-producer-server): Nexus IP provided explicitly, will use it addr = [fd00:1122:3344:101::a]:12221
23642025-08-26T21:47:02.067ZDEBGnexus (nexus-producer-server): starting producer registration task
23652025-08-26T21:47:02.067ZINFOnexus (nexus-producer-server): starting oximeter metric producer server address = [fd00:1122:3344:101::a]:35154 file = oximeter/producer/src/lib.rs:279 interval = 10s producer_id = 4996dcc4-c530-4f64-b407-fa372462bd9e
23662025-08-26T21:47:02.067ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0
23672025-08-26T21:47:02.067ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221
23682025-08-26T21:47:02.150ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
23692025-08-26T21:47:02.198ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
23702025-08-26T21:47:02.198ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
23712025-08-26T21:47:02.198ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "9c1d3209-4dc6-4620-8de3-f76535ed3d31", "date": "Tue, 26 Aug 2025 21:47:01 GMT"} })
23722025-08-26T21:47:02.198ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
23732025-08-26T21:47:02.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
23742025-08-26T21:47:02.202ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
23752025-08-26T21:47:02.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
23762025-08-26T21:47:02.204ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
23772025-08-26T21:47:02.204ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::a]:57490
23782025-08-26T21:47:02.209ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
23792025-08-26T21:47:02.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
23802025-08-26T21:47:02.212ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:57490 req_id = 50f62606-4e1b-48fe-bb03-4be2677e670a roles = RoleSet { roles: {} } uri = /metrics/producers
23812025-08-26T21:47:02.213ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:57490 req_id = 50f62606-4e1b-48fe-bb03-4be2677e670a resource = Database result = Ok(()) uri = /metrics/producers
23822025-08-26T21:47:02.213ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
23832025-08-26T21:47:02.213ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
23842025-08-26T21:47:02.213ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "2eb6d6ac-84db-422e-82db-487d4c447dcc", "date": "Tue, 26 Aug 2025 21:47:02 GMT"} })
23852025-08-26T21:47:02.213ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23862025-08-26T21:47:02.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23872025-08-26T21:47:02.214ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 525.949107ms iteration = 3 status = {}
23882025-08-26T21:47:02.217ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
23892025-08-26T21:47:02.217ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23902025-08-26T21:47:02.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23912025-08-26T21:47:02.220ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
23922025-08-26T21:47:02.248ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
23932025-08-26T21:47:02.248ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "336ea9b5-eb45-4bd5-a151-95950242c461", "date": "Tue, 26 Aug 2025 21:47:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
23942025-08-26T21:47:02.248ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
23952025-08-26T21:47:02.250ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "5fe73c1f-4f98-4761-86b6-0ee7314dea31", "date": "Tue, 26 Aug 2025 21:47:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
23962025-08-26T21:47:02.250ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
23972025-08-26T21:47:02.251ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "f32a9c35-2f58-41ba-b27d-9b95a7561c4a", "date": "Tue, 26 Aug 2025 21:47:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
23982025-08-26T21:47:02.251ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
23992025-08-26T21:47:02.252ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
24002025-08-26T21:47:02.252ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 317.414195ms iteration = 4 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
24012025-08-26T21:47:02.258ZINFOnexus: assigned collector to new producer collector_id = f843ff0e-eef8-49c4-b428-23e15df1def3 file = nexus/src/app/oximeter.rs:76 producer_id = 4996dcc4-c530-4f64-b407-fa372462bd9e
24022025-08-26T21:47:02.258ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 54328 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:57490 req_id = 50f62606-4e1b-48fe-bb03-4be2677e670a response_code = 201 uri = /metrics/producers
24032025-08-26T21:47:02.259ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "50f62606-4e1b-48fe-bb03-4be2677e670a", "content-length": "41", "date": "Tue, 26 Aug 2025 21:47:02 GMT"} })
24042025-08-26T21:47:02.261ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
24052025-08-26T21:47:02.269ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
24062025-08-26T21:47:02.269ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "5c9d5a95-34ce-46ba-b33c-04414de7d765", "date": "Tue, 26 Aug 2025 21:47:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
24072025-08-26T21:47:02.269ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
24082025-08-26T21:47:02.270ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "3dbad57e-86bb-4919-ab4a-08c2d2232fc4", "date": "Tue, 26 Aug 2025 21:47:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
24092025-08-26T21:47:02.270ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
24102025-08-26T21:47:02.271ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
24112025-08-26T21:47:02.271ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 290.963499ms iteration = 4 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
24122025-08-26T21:47:02.320ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
24132025-08-26T21:47:02.320ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
24142025-08-26T21:47:02.320ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
24152025-08-26T21:47:02.320ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
24162025-08-26T21:47:02.320ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
24172025-08-26T21:47:02.320ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
24182025-08-26T21:47:02.320ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
24192025-08-26T21:47:02.320ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
24202025-08-26T21:47:02.320ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
24212025-08-26T21:47:02.320ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
24222025-08-26T21:47:02.321ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
24232025-08-26T21:47:02.321ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24242025-08-26T21:47:02.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24252025-08-26T21:47:02.323ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
24262025-08-26T21:47:02.323ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
24272025-08-26T21:47:02.423ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "f673da9d-6101-4cb0-9e87-91853679cf6b", "date": "Tue, 26 Aug 2025 21:47:01 GMT"} })
24282025-08-26T21:47:02.423ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
24292025-08-26T21:47:02.423ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
24302025-08-26T21:47:02.429ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "0aa5607b-dd3b-46f9-a36a-94ef7ff6a9fc", "content-length": "682", "date": "Tue, 26 Aug 2025 21:47:02 GMT"} })
24312025-08-26T21:47:02.429ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24322025-08-26T21:47:02.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24332025-08-26T21:47:02.433ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24342025-08-26T21:47:02.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24352025-08-26T21:47:02.436ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24362025-08-26T21:47:02.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24372025-08-26T21:47:02.438ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24382025-08-26T21:47:02.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24392025-08-26T21:47:02.457ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 781.807593ms iteration = 3 status = {}
24402025-08-26T21:47:06.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 14 reason = Timeout
24412025-08-26T21:47:06.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
24422025-08-26T21:47:06.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
24432025-08-26T21:47:06.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24442025-08-26T21:47:06.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
24452025-08-26T21:47:06.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
24462025-08-26T21:47:06.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
24472025-08-26T21:47:06.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
24482025-08-26T21:47:06.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.684051ms iteration = 14 status = {"chicken_switches_updated":false}
24492025-08-26T21:47:11.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 15 reason = Timeout
24502025-08-26T21:47:11.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
24512025-08-26T21:47:11.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
24522025-08-26T21:47:11.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24532025-08-26T21:47:11.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
24542025-08-26T21:47:11.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
24552025-08-26T21:47:11.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
24562025-08-26T21:47:11.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
24572025-08-26T21:47:11.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.908235ms iteration = 15 status = {"chicken_switches_updated":false}
24582025-08-26T21:47:11.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 8 reason = Timeout
24592025-08-26T21:47:11.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24602025-08-26T21:47:11.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24612025-08-26T21:47:11.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24622025-08-26T21:47:11.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
24632025-08-26T21:47:11.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24642025-08-26T21:47:11.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24652025-08-26T21:47:11.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24662025-08-26T21:47:11.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24672025-08-26T21:47:11.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24682025-08-26T21:47:11.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 8 reason = Timeout
24692025-08-26T21:47:11.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24702025-08-26T21:47:11.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
24712025-08-26T21:47:11.688ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
24722025-08-26T21:47:11.688ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24732025-08-26T21:47:11.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
24742025-08-26T21:47:11.704ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
24752025-08-26T21:47:11.704ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.240999ms iteration = 8 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
24762025-08-26T21:47:11.729ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24772025-08-26T21:47:11.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
24782025-08-26T21:47:11.730ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 44.295499ms iteration = 8 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
24792025-08-26T21:47:15.344ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:42215
24802025-08-26T21:47:15.344ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42215 req_id = f3f62740-6408-4a23-8e7f-a640f395e81b roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
24812025-08-26T21:47:15.345ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42215 req_id = f3f62740-6408-4a23-8e7f-a640f395e81b resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
24822025-08-26T21:47:15.347ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2963 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42215 req_id = f3f62740-6408-4a23-8e7f-a640f395e81b response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
24832025-08-26T21:47:15.348ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42215 req_id = 0d5dc449-ebbb-47e7-861c-040fbd2730f9 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
24842025-08-26T21:47:15.348ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42215 req_id = 0d5dc449-ebbb-47e7-861c-040fbd2730f9 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
24852025-08-26T21:47:15.350ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2276 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42215 req_id = 0d5dc449-ebbb-47e7-861c-040fbd2730f9 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
24862025-08-26T21:47:15.658ZINFOnexus (dropshot): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:35154 remote_addr = [fd00:1122:3344:101::d]:59465
24872025-08-26T21:47:15.670ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 11376 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 321a0a8d-1b78-4c5b-afb4-714da6951321 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
24882025-08-26T21:47:16.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 16 reason = Timeout
24892025-08-26T21:47:16.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
24902025-08-26T21:47:16.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
24912025-08-26T21:47:16.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24922025-08-26T21:47:16.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
24932025-08-26T21:47:16.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
24942025-08-26T21:47:16.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
24952025-08-26T21:47:16.672ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
24962025-08-26T21:47:16.672ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.528607ms iteration = 16 status = {"chicken_switches_updated":false}
24972025-08-26T21:47:21.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 17 reason = Timeout
24982025-08-26T21:47:21.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
24992025-08-26T21:47:21.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
25002025-08-26T21:47:21.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25012025-08-26T21:47:21.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
25022025-08-26T21:47:21.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
25032025-08-26T21:47:21.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
25042025-08-26T21:47:21.675ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
25052025-08-26T21:47:21.675ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.817086ms iteration = 17 status = {"chicken_switches_updated":false}
25062025-08-26T21:47:21.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 9 reason = Timeout
25072025-08-26T21:47:21.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25082025-08-26T21:47:21.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25092025-08-26T21:47:21.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25102025-08-26T21:47:21.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
25112025-08-26T21:47:21.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25122025-08-26T21:47:21.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25132025-08-26T21:47:21.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25142025-08-26T21:47:21.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25152025-08-26T21:47:21.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25162025-08-26T21:47:21.687ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
25172025-08-26T21:47:21.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25182025-08-26T21:47:21.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25192025-08-26T21:47:21.688ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 9 reason = Timeout
25202025-08-26T21:47:21.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
25212025-08-26T21:47:21.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
25222025-08-26T21:47:21.690ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
25232025-08-26T21:47:21.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
25242025-08-26T21:47:21.693ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.941032ms iteration = 9 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
25252025-08-26T21:47:21.708ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
25262025-08-26T21:47:21.708ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 33.248939ms iteration = 9 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
25272025-08-26T21:47:25.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 345 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = e82e71d5-3e72-44a6-9863-40d3968cb2dd response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
25282025-08-26T21:47:26.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 18 reason = Timeout
25292025-08-26T21:47:26.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
25302025-08-26T21:47:26.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
25312025-08-26T21:47:26.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25322025-08-26T21:47:26.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
25332025-08-26T21:47:26.673ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
25342025-08-26T21:47:26.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
25352025-08-26T21:47:26.675ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
25362025-08-26T21:47:26.675ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.587111ms iteration = 18 status = {"chicken_switches_updated":false}
25372025-08-26T21:47:31.666ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 4 reason = Timeout
25382025-08-26T21:47:31.666ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 4 reason = Timeout
25392025-08-26T21:47:31.666ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
25402025-08-26T21:47:31.666ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 4 reason = Timeout
25412025-08-26T21:47:31.666ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
25422025-08-26T21:47:31.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
25432025-08-26T21:47:31.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
25442025-08-26T21:47:31.668ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 19 reason = Timeout
25452025-08-26T21:47:31.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
25462025-08-26T21:47:31.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
25472025-08-26T21:47:31.668ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 955, tv_nsec: 525892868 } })
25482025-08-26T21:47:31.668ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
25492025-08-26T21:47:31.669ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 955, tv_nsec: 526711926 } })
25502025-08-26T21:47:31.669ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
25512025-08-26T21:47:31.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25522025-08-26T21:47:31.671ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.556615ms iteration = 4 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
25532025-08-26T21:47:31.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
25542025-08-26T21:47:31.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
25552025-08-26T21:47:31.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
25562025-08-26T21:47:31.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
25572025-08-26T21:47:31.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.751902ms iteration = 19 status = {"chicken_switches_updated":false}
25582025-08-26T21:47:31.675ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 5 reason = Timeout
25592025-08-26T21:47:31.675ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 4 reason = Timeout
25602025-08-26T21:47:31.675ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25612025-08-26T21:47:31.675ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25622025-08-26T21:47:31.675ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 4 reason = Timeout
25632025-08-26T21:47:31.675ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25642025-08-26T21:47:31.675ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 4 reason = Timeout
25652025-08-26T21:47:31.675ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
25662025-08-26T21:47:31.675ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 4 reason = Timeout
25672025-08-26T21:47:31.675ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
25682025-08-26T21:47:31.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 10 reason = Timeout
25692025-08-26T21:47:31.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25702025-08-26T21:47:31.675ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 4 reason = Timeout
25712025-08-26T21:47:31.675ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25722025-08-26T21:47:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25732025-08-26T21:47:31.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
25742025-08-26T21:47:31.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
25752025-08-26T21:47:31.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25762025-08-26T21:47:31.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
25772025-08-26T21:47:31.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25782025-08-26T21:47:31.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25792025-08-26T21:47:31.677ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25802025-08-26T21:47:31.677ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25812025-08-26T21:47:31.677ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25822025-08-26T21:47:31.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25832025-08-26T21:47:31.678ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25842025-08-26T21:47:31.680ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
25852025-08-26T21:47:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
25862025-08-26T21:47:31.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
25872025-08-26T21:47:31.682ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25882025-08-26T21:47:31.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
25892025-08-26T21:47:31.682ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25902025-08-26T21:47:31.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25912025-08-26T21:47:31.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25922025-08-26T21:47:31.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
25932025-08-26T21:47:31.683ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
25942025-08-26T21:47:31.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
25952025-08-26T21:47:31.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
25962025-08-26T21:47:31.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25972025-08-26T21:47:31.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
25982025-08-26T21:47:31.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
25992025-08-26T21:47:31.685ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26002025-08-26T21:47:31.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26012025-08-26T21:47:31.686ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 10.64116ms iteration = 4 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
26022025-08-26T21:47:31.687ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26032025-08-26T21:47:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26042025-08-26T21:47:31.687ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 4 reason = Timeout
26052025-08-26T21:47:31.688ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
26062025-08-26T21:47:31.688ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 4 reason = Timeout
26072025-08-26T21:47:31.688ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
26082025-08-26T21:47:31.688ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
26092025-08-26T21:47:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
26102025-08-26T21:47:31.688ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 10 reason = Timeout
26112025-08-26T21:47:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
26122025-08-26T21:47:31.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
26132025-08-26T21:47:31.688ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 4 reason = Timeout
26142025-08-26T21:47:31.688ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 8.67\u{b5}s iteration = 4 status = {"disabled":true,"errors":[],"sps":[]}
26152025-08-26T21:47:31.688ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 4 reason = Timeout
26162025-08-26T21:47:31.688ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
26172025-08-26T21:47:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26182025-08-26T21:47:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
26192025-08-26T21:47:31.688ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 4 reason = Timeout
26202025-08-26T21:47:31.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
26212025-08-26T21:47:31.688ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 4 reason = Timeout
26222025-08-26T21:47:31.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
26232025-08-26T21:47:31.688ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 4 reason = Timeout
26242025-08-26T21:47:31.689ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
26252025-08-26T21:47:31.689ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26262025-08-26T21:47:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
26272025-08-26T21:47:31.689ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 4 reason = Timeout
26282025-08-26T21:47:31.689ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
26292025-08-26T21:47:31.689ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 4 reason = Timeout
26302025-08-26T21:47:31.689ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
26312025-08-26T21:47:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
26322025-08-26T21:47:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26332025-08-26T21:47:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
26342025-08-26T21:47:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
26352025-08-26T21:47:31.690ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
26362025-08-26T21:47:31.690ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 955, tv_nsec: 547882993 } })
26372025-08-26T21:47:31.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
26382025-08-26T21:47:31.691ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
26392025-08-26T21:47:31.691ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26402025-08-26T21:47:31.692ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 3.068949ms iteration = 4 status = {"errors":[],"garbage_collect_requested":[]}
26412025-08-26T21:47:31.692ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
26422025-08-26T21:47:31.692ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
26432025-08-26T21:47:31.692ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
26442025-08-26T21:47:31.692ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
26452025-08-26T21:47:31.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
26462025-08-26T21:47:31.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
26472025-08-26T21:47:31.694ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26482025-08-26T21:47:31.694ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
26492025-08-26T21:47:31.694ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26502025-08-26T21:47:31.694ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26512025-08-26T21:47:31.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26522025-08-26T21:47:31.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26532025-08-26T21:47:31.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26542025-08-26T21:47:31.695ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
26552025-08-26T21:47:31.695ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26562025-08-26T21:47:31.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
26572025-08-26T21:47:31.695ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
26582025-08-26T21:47:31.695ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
26592025-08-26T21:47:31.695ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
26602025-08-26T21:47:31.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
26612025-08-26T21:47:31.696ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26622025-08-26T21:47:31.698ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
26632025-08-26T21:47:31.698ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
26642025-08-26T21:47:31.698ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 10.307563ms iteration = 4 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
26652025-08-26T21:47:31.698ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
26662025-08-26T21:47:31.698ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
26672025-08-26T21:47:31.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
26682025-08-26T21:47:31.700ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26692025-08-26T21:47:31.700ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26702025-08-26T21:47:31.701ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
26712025-08-26T21:47:31.701ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26722025-08-26T21:47:31.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
26732025-08-26T21:47:31.707ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
26742025-08-26T21:47:31.707ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26752025-08-26T21:47:31.707ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
26762025-08-26T21:47:31.707ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
26772025-08-26T21:47:31.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26782025-08-26T21:47:31.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
26792025-08-26T21:47:31.709ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26802025-08-26T21:47:31.712ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
26812025-08-26T21:47:31.712ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26822025-08-26T21:47:31.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26832025-08-26T21:47:31.727ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
26842025-08-26T21:47:31.727ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 39.105791ms iteration = 4 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
26852025-08-26T21:47:31.739ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
26862025-08-26T21:47:31.740ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 64.276628ms iteration = 10 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
26872025-08-26T21:47:31.759ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
26882025-08-26T21:47:31.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
26892025-08-26T21:47:31.760ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
26902025-08-26T21:47:31.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26912025-08-26T21:47:31.763ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 74.795766ms iteration = 10 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
26922025-08-26T21:47:31.767ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26932025-08-26T21:47:31.768ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 79.316358ms iteration = 4 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
26942025-08-26T21:47:31.768ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 79.597715ms iteration = 4 status = {"errors":[],"requests_created_ok":[]}
26952025-08-26T21:47:31.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26962025-08-26T21:47:31.770ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26972025-08-26T21:47:31.770ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 81.712642ms iteration = 4 status = {"errors":[],"finish_invoked_ok":[]}
26982025-08-26T21:47:31.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26992025-08-26T21:47:31.772ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27002025-08-26T21:47:31.777ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
27012025-08-26T21:47:31.777ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 101.680682ms iteration = 5 status = {"physical_disks_added":0}
27022025-08-26T21:47:31.777ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27032025-08-26T21:47:31.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27042025-08-26T21:47:31.778ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
27052025-08-26T21:47:31.778ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 102.325667ms iteration = 4 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
27062025-08-26T21:47:31.779ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 103.875212ms iteration = 4 status = 0
27072025-08-26T21:47:31.780ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
27082025-08-26T21:47:31.780ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
27092025-08-26T21:47:31.781ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff527af9-fb20-412f-a932-7266dc1beb4b", "content-length": "26", "date": "Tue, 26 Aug 2025 21:47:31 GMT"} })
27102025-08-26T21:47:31.781ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
27112025-08-26T21:47:31.782ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
27122025-08-26T21:47:31.813ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 124.318923ms iteration = 4 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
27132025-08-26T21:47:31.821ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
27142025-08-26T21:47:31.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
27152025-08-26T21:47:31.833ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
27162025-08-26T21:47:31.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
27172025-08-26T21:47:31.835ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27182025-08-26T21:47:31.837ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
27192025-08-26T21:47:31.837ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
27202025-08-26T21:47:31.837ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
27212025-08-26T21:47:31.837ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
27222025-08-26T21:47:31.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
27232025-08-26T21:47:31.838ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c8766dd-87d2-4b48-a82b-e2552c0bf24e", "content-length": "26", "date": "Tue, 26 Aug 2025 21:47:31 GMT"} })
27242025-08-26T21:47:31.838ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
27252025-08-26T21:47:31.839ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
27262025-08-26T21:47:31.839ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27272025-08-26T21:47:31.841ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
27282025-08-26T21:47:31.841ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
27292025-08-26T21:47:31.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
27302025-08-26T21:47:31.854ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
27312025-08-26T21:47:31.854ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
27322025-08-26T21:47:31.854ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "0330667f-648a-40f8-a57a-1b6a19fd6f44", "content-length": "372", "date": "Tue, 26 Aug 2025 21:47:31 GMT"} })
27332025-08-26T21:47:31.855ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27342025-08-26T21:47:31.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27352025-08-26T21:47:31.871ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
27362025-08-26T21:47:31.871ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
27372025-08-26T21:47:31.871ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "0f63d172-221a-43af-a32a-fbd53cebf93b", "date": "Tue, 26 Aug 2025 21:47:31 GMT"} })
27382025-08-26T21:47:31.872ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27392025-08-26T21:47:31.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27402025-08-26T21:47:31.875ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27412025-08-26T21:47:31.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27422025-08-26T21:47:31.879ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27432025-08-26T21:47:31.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27442025-08-26T21:47:31.882ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
27452025-08-26T21:47:31.882ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
27462025-08-26T21:47:31.883ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "74d33cd9-41cb-40b2-b7dd-c2079e4c6d40", "date": "Tue, 26 Aug 2025 21:47:31 GMT"} })
27472025-08-26T21:47:31.884ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 195.630799ms iteration = 4 status = {}
27482025-08-26T21:47:31.891ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 225.025783ms iteration = 4 status = {}
27492025-08-26T21:47:31.927ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
27502025-08-26T21:47:31.927ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
27512025-08-26T21:47:31.928ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "055c47e5-7751-4c84-bbdb-7fa2a8fca90a", "content-length": "26", "date": "Tue, 26 Aug 2025 21:47:31 GMT"} })
27522025-08-26T21:47:31.928ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
27532025-08-26T21:47:31.929ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
27542025-08-26T21:47:31.934ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
27552025-08-26T21:47:31.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
27562025-08-26T21:47:31.936ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 270.648131ms iteration = 4 status = null
27572025-08-26T21:47:31.960ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
27582025-08-26T21:47:31.961ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b208883-3581-4d0a-b70c-3c26adb8c57a", "content-length": "691", "date": "Tue, 26 Aug 2025 21:47:31 GMT"} })
27592025-08-26T21:47:31.961ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
27602025-08-26T21:47:31.961ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
27612025-08-26T21:47:31.961ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
27622025-08-26T21:47:31.963ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 287.496501ms iteration = 4 status = {}
27632025-08-26T21:47:32.036ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27642025-08-26T21:47:32.036ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27652025-08-26T21:47:32.038ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
27662025-08-26T21:47:32.038ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27672025-08-26T21:47:32.038ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27682025-08-26T21:47:32.150ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
27692025-08-26T21:47:32.150ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
27702025-08-26T21:47:32.150ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
27712025-08-26T21:47:32.150ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
27722025-08-26T21:47:32.150ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
27732025-08-26T21:47:32.150ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
27742025-08-26T21:47:32.150ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
27752025-08-26T21:47:32.150ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
27762025-08-26T21:47:32.151ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
27772025-08-26T21:47:32.151ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
27782025-08-26T21:47:32.151ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
27792025-08-26T21:47:32.151ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27802025-08-26T21:47:32.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27812025-08-26T21:47:32.153ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
27822025-08-26T21:47:32.153ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
27832025-08-26T21:47:32.290ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "8d207f4f-2676-4952-a2a9-37caea99c169", "date": "Tue, 26 Aug 2025 21:47:31 GMT"} })
27842025-08-26T21:47:32.290ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
27852025-08-26T21:47:32.290ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
27862025-08-26T21:47:32.291ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d793f50-6265-45c8-ad94-11c5815b8880", "content-length": "682", "date": "Tue, 26 Aug 2025 21:47:31 GMT"} })
27872025-08-26T21:47:32.291ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27882025-08-26T21:47:32.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27892025-08-26T21:47:32.314ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27902025-08-26T21:47:32.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27912025-08-26T21:47:32.318ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27922025-08-26T21:47:32.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27932025-08-26T21:47:32.320ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27942025-08-26T21:47:32.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27952025-08-26T21:47:32.327ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 651.505656ms iteration = 4 status = {}
27962025-08-26T21:47:35.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 336 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 9a8de698-2d10-48be-aa30-6a009c323908 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
27972025-08-26T21:47:36.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 20 reason = Timeout
27982025-08-26T21:47:36.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
27992025-08-26T21:47:36.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
28002025-08-26T21:47:36.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28012025-08-26T21:47:36.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
28022025-08-26T21:47:36.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
28032025-08-26T21:47:36.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
28042025-08-26T21:47:36.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
28052025-08-26T21:47:36.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.365786ms iteration = 20 status = {"chicken_switches_updated":false}
28062025-08-26T21:47:41.668ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 21 reason = Timeout
28072025-08-26T21:47:41.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
28082025-08-26T21:47:41.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
28092025-08-26T21:47:41.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28102025-08-26T21:47:41.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
28112025-08-26T21:47:41.673ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
28122025-08-26T21:47:41.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
28132025-08-26T21:47:41.675ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
28142025-08-26T21:47:41.675ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.473168ms iteration = 21 status = {"chicken_switches_updated":false}
28152025-08-26T21:47:41.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 11 reason = Timeout
28162025-08-26T21:47:41.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28172025-08-26T21:47:41.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28182025-08-26T21:47:41.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28192025-08-26T21:47:41.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
28202025-08-26T21:47:41.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28212025-08-26T21:47:41.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28222025-08-26T21:47:41.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28232025-08-26T21:47:41.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28242025-08-26T21:47:41.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28252025-08-26T21:47:41.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
28262025-08-26T21:47:41.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28272025-08-26T21:47:41.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28282025-08-26T21:47:41.687ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 11 reason = Timeout
28292025-08-26T21:47:41.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28302025-08-26T21:47:41.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28312025-08-26T21:47:41.689ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28322025-08-26T21:47:41.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28332025-08-26T21:47:41.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.10146ms iteration = 11 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
28342025-08-26T21:47:41.699ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
28352025-08-26T21:47:41.699ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.266087ms iteration = 11 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
28362025-08-26T21:47:45.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 330 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 831909fd-9106-47d7-9e61-452a2b1fbec1 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
28372025-08-26T21:47:46.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 22 reason = Timeout
28382025-08-26T21:47:46.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
28392025-08-26T21:47:46.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
28402025-08-26T21:47:46.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28412025-08-26T21:47:46.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
28422025-08-26T21:47:46.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
28432025-08-26T21:47:46.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
28442025-08-26T21:47:46.675ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
28452025-08-26T21:47:46.675ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.190182ms iteration = 22 status = {"chicken_switches_updated":false}
28462025-08-26T21:47:51.668ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 23 reason = Timeout
28472025-08-26T21:47:51.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
28482025-08-26T21:47:51.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
28492025-08-26T21:47:51.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28502025-08-26T21:47:51.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
28512025-08-26T21:47:51.673ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
28522025-08-26T21:47:51.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
28532025-08-26T21:47:51.675ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
28542025-08-26T21:47:51.675ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.476368ms iteration = 23 status = {"chicken_switches_updated":false}
28552025-08-26T21:47:51.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 12 reason = Timeout
28562025-08-26T21:47:51.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28572025-08-26T21:47:51.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28582025-08-26T21:47:51.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28592025-08-26T21:47:51.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
28602025-08-26T21:47:51.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28612025-08-26T21:47:51.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28622025-08-26T21:47:51.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28632025-08-26T21:47:51.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28642025-08-26T21:47:51.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28652025-08-26T21:47:51.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
28662025-08-26T21:47:51.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28672025-08-26T21:47:51.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
28682025-08-26T21:47:51.687ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 12 reason = Timeout
28692025-08-26T21:47:51.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28702025-08-26T21:47:51.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28712025-08-26T21:47:51.689ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28722025-08-26T21:47:51.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28732025-08-26T21:47:51.691ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.595821ms iteration = 12 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
28742025-08-26T21:47:51.698ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
28752025-08-26T21:47:51.698ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.50505ms iteration = 12 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
28762025-08-26T21:47:55.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 349 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 4ea0e140-c4e7-4a36-b212-253c21ab2f3d response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
28772025-08-26T21:47:56.668ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 24 reason = Timeout
28782025-08-26T21:47:56.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
28792025-08-26T21:47:56.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
28802025-08-26T21:47:56.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28812025-08-26T21:47:56.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
28822025-08-26T21:47:56.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
28832025-08-26T21:47:56.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
28842025-08-26T21:47:56.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
28852025-08-26T21:47:56.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.798223ms iteration = 24 status = {"chicken_switches_updated":false}
28862025-08-26T21:48:00.346ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:56248
28872025-08-26T21:48:00.346ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56248 req_id = aa708f62-a6f4-4e5a-9ee6-6a6dce78fd1b roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
28882025-08-26T21:48:00.346ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56248 req_id = aa708f62-a6f4-4e5a-9ee6-6a6dce78fd1b resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
28892025-08-26T21:48:00.350ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3725 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56248 req_id = aa708f62-a6f4-4e5a-9ee6-6a6dce78fd1b response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
28902025-08-26T21:48:00.350ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56248 req_id = 4246efe8-df68-4f6a-9bd5-6f7164421dfd roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
28912025-08-26T21:48:00.350ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56248 req_id = 4246efe8-df68-4f6a-9bd5-6f7164421dfd resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
28922025-08-26T21:48:00.352ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2487 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56248 req_id = 4246efe8-df68-4f6a-9bd5-6f7164421dfd response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
28932025-08-26T21:48:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 5 reason = Timeout
28942025-08-26T21:48:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 3 reason = Timeout
28952025-08-26T21:48:01.665ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
28962025-08-26T21:48:01.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
28972025-08-26T21:48:01.665ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 5 reason = Timeout
28982025-08-26T21:48:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 3 reason = Timeout
28992025-08-26T21:48:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 5 reason = Timeout
29002025-08-26T21:48:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 3 reason = Timeout
29012025-08-26T21:48:01.665ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
29022025-08-26T21:48:01.665ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 3 reason = Timeout
29032025-08-26T21:48:01.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
29042025-08-26T21:48:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 3 reason = Timeout
29052025-08-26T21:48:01.665ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
29062025-08-26T21:48:01.666ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 5 reason = Timeout
29072025-08-26T21:48:01.666ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
29082025-08-26T21:48:01.666ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 5 reason = Timeout
29092025-08-26T21:48:01.666ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 3 reason = Timeout
29102025-08-26T21:48:01.666ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
29112025-08-26T21:48:01.666ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:38:01.665999676 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
29122025-08-26T21:48:01.666ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
29132025-08-26T21:48:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
29142025-08-26T21:48:01.666ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 4 reason = Timeout
29152025-08-26T21:48:01.666ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
29162025-08-26T21:48:01.666ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 18.38\u{b5}s iteration = 4 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
29172025-08-26T21:48:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
29182025-08-26T21:48:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
29192025-08-26T21:48:01.666ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 985, tv_nsec: 523803431 } })
29202025-08-26T21:48:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
29212025-08-26T21:48:01.666ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 985, tv_nsec: 523813951 } })
29222025-08-26T21:48:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
29232025-08-26T21:48:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
29242025-08-26T21:48:01.666ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
29252025-08-26T21:48:01.666ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.013623ms iteration = 3 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
29262025-08-26T21:48:01.666ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
29272025-08-26T21:48:01.666ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
29282025-08-26T21:48:01.667ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
29292025-08-26T21:48:01.667ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.051254ms iteration = 3 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
29302025-08-26T21:48:01.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
29312025-08-26T21:48:01.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
29322025-08-26T21:48:01.667ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:38:01.665999676 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
29332025-08-26T21:48:01.668ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 1.984574ms iteration = 3 status = {"expiration":"2025-08-26T21:38:01.665999676Z","pruned":[]}
29342025-08-26T21:48:01.668ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29352025-08-26T21:48:01.668ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 25 reason = Timeout
29362025-08-26T21:48:01.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
29372025-08-26T21:48:01.668ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 985, tv_nsec: 525832896 } })
29382025-08-26T21:48:01.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
29392025-08-26T21:48:01.668ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
29402025-08-26T21:48:01.668ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 985, tv_nsec: 526309577 } })
29412025-08-26T21:48:01.669ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
29422025-08-26T21:48:01.669ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29432025-08-26T21:48:01.669ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.439968ms iteration = 5 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
29442025-08-26T21:48:01.669ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29452025-08-26T21:48:01.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29462025-08-26T21:48:01.670ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
29472025-08-26T21:48:01.670ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
29482025-08-26T21:48:01.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
29492025-08-26T21:48:01.672ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
29502025-08-26T21:48:01.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
29512025-08-26T21:48:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
29522025-08-26T21:48:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
29532025-08-26T21:48:01.674ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
29542025-08-26T21:48:01.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 5 reason = Timeout
29552025-08-26T21:48:01.674ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
29562025-08-26T21:48:01.674ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 10.47\u{b5}s iteration = 5 status = "Disabled"
29572025-08-26T21:48:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
29582025-08-26T21:48:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
29592025-08-26T21:48:01.674ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29602025-08-26T21:48:01.675ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 3 reason = Timeout
29612025-08-26T21:48:01.675ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
29622025-08-26T21:48:01.675ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 5 reason = Timeout
29632025-08-26T21:48:01.675ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
29642025-08-26T21:48:01.675ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
29652025-08-26T21:48:01.675ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 5 reason = Timeout
29662025-08-26T21:48:01.675ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
29672025-08-26T21:48:01.675ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 5 reason = Timeout
29682025-08-26T21:48:01.675ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29692025-08-26T21:48:01.675ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 6 reason = Timeout
29702025-08-26T21:48:01.675ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
29712025-08-26T21:48:01.675ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 5 reason = Timeout
29722025-08-26T21:48:01.675ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
29732025-08-26T21:48:01.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 13 reason = Timeout
29742025-08-26T21:48:01.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
29752025-08-26T21:48:01.675ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 5 reason = Timeout
29762025-08-26T21:48:01.675ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
29772025-08-26T21:48:01.675ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
29782025-08-26T21:48:01.675ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
29792025-08-26T21:48:01.675ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
29802025-08-26T21:48:01.675ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
29812025-08-26T21:48:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
29822025-08-26T21:48:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
29832025-08-26T21:48:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
29842025-08-26T21:48:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
29852025-08-26T21:48:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
29862025-08-26T21:48:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
29872025-08-26T21:48:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
29882025-08-26T21:48:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
29892025-08-26T21:48:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
29902025-08-26T21:48:01.677ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
29912025-08-26T21:48:01.677ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.961772ms iteration = 25 status = {"chicken_switches_updated":false}
29922025-08-26T21:48:01.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29932025-08-26T21:48:01.678ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29942025-08-26T21:48:01.678ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29952025-08-26T21:48:01.678ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29962025-08-26T21:48:01.678ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29972025-08-26T21:48:01.678ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29982025-08-26T21:48:01.679ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29992025-08-26T21:48:01.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
30002025-08-26T21:48:01.681ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
30012025-08-26T21:48:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
30022025-08-26T21:48:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
30032025-08-26T21:48:01.683ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
30042025-08-26T21:48:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
30052025-08-26T21:48:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
30062025-08-26T21:48:01.684ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
30072025-08-26T21:48:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
30082025-08-26T21:48:01.684ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
30092025-08-26T21:48:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
30102025-08-26T21:48:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
30112025-08-26T21:48:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
30122025-08-26T21:48:01.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30132025-08-26T21:48:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30142025-08-26T21:48:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
30152025-08-26T21:48:01.685ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30162025-08-26T21:48:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30172025-08-26T21:48:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
30182025-08-26T21:48:01.685ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
30192025-08-26T21:48:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
30202025-08-26T21:48:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
30212025-08-26T21:48:01.686ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
30222025-08-26T21:48:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
30232025-08-26T21:48:01.687ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 3 reason = Timeout
30242025-08-26T21:48:01.687ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
30252025-08-26T21:48:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
30262025-08-26T21:48:01.688ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 5 reason = Timeout
30272025-08-26T21:48:01.688ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
30282025-08-26T21:48:01.688ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
30292025-08-26T21:48:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
30302025-08-26T21:48:01.688ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 5 reason = Timeout
30312025-08-26T21:48:01.688ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
30322025-08-26T21:48:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
30332025-08-26T21:48:01.688ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 13 reason = Timeout
30342025-08-26T21:48:01.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
30352025-08-26T21:48:01.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
30362025-08-26T21:48:01.689ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 3 reason = Timeout
30372025-08-26T21:48:01.689ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
30382025-08-26T21:48:01.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
30392025-08-26T21:48:01.689ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 3 reason = Timeout
30402025-08-26T21:48:01.689ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
30412025-08-26T21:48:01.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
30422025-08-26T21:48:01.690ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 3 reason = Timeout
30432025-08-26T21:48:01.690ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
30442025-08-26T21:48:01.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
30452025-08-26T21:48:01.690ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 3 reason = Timeout
30462025-08-26T21:48:01.690ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
30472025-08-26T21:48:01.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
30482025-08-26T21:48:01.690ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 5 reason = Timeout
30492025-08-26T21:48:01.690ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 7.731\u{b5}s iteration = 5 status = {"disabled":true,"errors":[],"sps":[]}
30502025-08-26T21:48:01.690ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 5 reason = Timeout
30512025-08-26T21:48:01.690ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
30522025-08-26T21:48:01.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
30532025-08-26T21:48:01.690ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 5 reason = Timeout
30542025-08-26T21:48:01.690ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
30552025-08-26T21:48:01.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
30562025-08-26T21:48:01.691ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 5 reason = Timeout
30572025-08-26T21:48:01.691ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
30582025-08-26T21:48:01.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
30592025-08-26T21:48:01.691ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 5 reason = Timeout
30602025-08-26T21:48:01.691ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
30612025-08-26T21:48:01.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
30622025-08-26T21:48:01.691ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 5 reason = Timeout
30632025-08-26T21:48:01.691ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
30642025-08-26T21:48:01.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
30652025-08-26T21:48:01.691ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 5 reason = Timeout
30662025-08-26T21:48:01.691ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
30672025-08-26T21:48:01.691ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30682025-08-26T21:48:01.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30692025-08-26T21:48:01.692ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
30702025-08-26T21:48:01.693ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30712025-08-26T21:48:01.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
30722025-08-26T21:48:01.693ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
30732025-08-26T21:48:01.693ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
30742025-08-26T21:48:01.693ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
30752025-08-26T21:48:01.693ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 985, tv_nsec: 551378102 } })
30762025-08-26T21:48:01.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
30772025-08-26T21:48:01.694ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
30782025-08-26T21:48:01.694ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 4.108853ms iteration = 5 status = {"errors":[],"garbage_collect_requested":[]}
30792025-08-26T21:48:01.695ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
30802025-08-26T21:48:01.695ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
30812025-08-26T21:48:01.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
30822025-08-26T21:48:01.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
30832025-08-26T21:48:01.697ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
30842025-08-26T21:48:01.697ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
30852025-08-26T21:48:01.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
30862025-08-26T21:48:01.697ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
30872025-08-26T21:48:01.698ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
30882025-08-26T21:48:01.698ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
30892025-08-26T21:48:01.698ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 7.897078ms iteration = 3 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
30902025-08-26T21:48:01.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
30912025-08-26T21:48:01.698ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
30922025-08-26T21:48:01.698ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
30932025-08-26T21:48:01.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
30942025-08-26T21:48:01.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
30952025-08-26T21:48:01.699ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 10.17281ms iteration = 3 status = {"errors":[],"found_port_ok":[]}
30962025-08-26T21:48:01.701ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
30972025-08-26T21:48:01.701ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 12.269867ms iteration = 3 status = {"by_rx":{},"error":null}
30982025-08-26T21:48:01.701ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30992025-08-26T21:48:01.702ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31002025-08-26T21:48:01.704ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
31012025-08-26T21:48:01.704ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 15.714445ms iteration = 13 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
31022025-08-26T21:48:01.704ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 14.354534ms iteration = 5 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
31032025-08-26T21:48:01.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
31042025-08-26T21:48:01.705ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31052025-08-26T21:48:01.705ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 15.147511ms iteration = 5 status = {"errors":[],"requests_created_ok":[]}
31062025-08-26T21:48:01.709ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31072025-08-26T21:48:01.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
31082025-08-26T21:48:01.709ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
31092025-08-26T21:48:01.709ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31102025-08-26T21:48:01.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
31112025-08-26T21:48:01.709ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
31122025-08-26T21:48:01.709ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
31132025-08-26T21:48:01.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
31142025-08-26T21:48:01.712ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
31152025-08-26T21:48:01.712ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 24.101193ms iteration = 5 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
31162025-08-26T21:48:01.712ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
31172025-08-26T21:48:01.712ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
31182025-08-26T21:48:01.713ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
31192025-08-26T21:48:01.745ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 55.856369ms iteration = 3 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
31202025-08-26T21:48:01.750ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
31212025-08-26T21:48:01.750ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 75.285917ms iteration = 13 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
31222025-08-26T21:48:01.757ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
31232025-08-26T21:48:01.758ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
31242025-08-26T21:48:01.758ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
31252025-08-26T21:48:01.758ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
31262025-08-26T21:48:01.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31272025-08-26T21:48:01.760ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31282025-08-26T21:48:01.765ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
31292025-08-26T21:48:01.765ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
31302025-08-26T21:48:01.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31312025-08-26T21:48:01.768ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
31322025-08-26T21:48:01.768ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
31332025-08-26T21:48:01.768ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
31342025-08-26T21:48:01.769ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
31352025-08-26T21:48:01.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31362025-08-26T21:48:01.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
31372025-08-26T21:48:01.770ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31382025-08-26T21:48:01.772ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 96.456054ms iteration = 5 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
31392025-08-26T21:48:01.772ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
31402025-08-26T21:48:01.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
31412025-08-26T21:48:01.773ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 81.863535ms iteration = 5 status = {"errors":[],"finish_invoked_ok":[]}
31422025-08-26T21:48:01.775ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
31432025-08-26T21:48:01.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
31442025-08-26T21:48:01.776ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
31452025-08-26T21:48:01.776ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
31462025-08-26T21:48:01.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
31472025-08-26T21:48:01.777ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
31482025-08-26T21:48:01.777ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 0 version = 2
31492025-08-26T21:48:01.777ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31502025-08-26T21:48:01.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31512025-08-26T21:48:01.779ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
31522025-08-26T21:48:01.779ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
31532025-08-26T21:48:01.779ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
31542025-08-26T21:48:01.779ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-08-26 21:46:04.065217 UTC dns_group = internal generation = 1
31552025-08-26T21:48:01.779ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 113.468078ms iteration = 3 status = {"generation":1}
31562025-08-26T21:48:01.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
31572025-08-26T21:48:01.779ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31582025-08-26T21:48:01.780ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
31592025-08-26T21:48:01.780ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
31602025-08-26T21:48:01.780ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
31612025-08-26T21:48:01.780ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31622025-08-26T21:48:01.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
31632025-08-26T21:48:01.781ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31642025-08-26T21:48:01.783ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
31652025-08-26T21:48:01.783ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31662025-08-26T21:48:01.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31672025-08-26T21:48:01.785ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
31682025-08-26T21:48:01.785ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
31692025-08-26T21:48:01.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
31702025-08-26T21:48:01.785ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31712025-08-26T21:48:01.785ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
31722025-08-26T21:48:01.786ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
31732025-08-26T21:48:01.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
31742025-08-26T21:48:01.786ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-08-26T21:45:59.866217Z, time_modified: 2025-08-26T21:45:59.866217Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }) }
31752025-08-26T21:48:01.786ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 120.761603ms iteration = 3 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]}},"default_endpoint":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
31762025-08-26T21:48:01.788ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
31772025-08-26T21:48:01.788ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31782025-08-26T21:48:01.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31792025-08-26T21:48:01.798ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 4 version = 2
31802025-08-26T21:48:01.798ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
31812025-08-26T21:48:01.798ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-08-26 21:46:05.296205 UTC dns_group = external generation = 2
31822025-08-26T21:48:01.798ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 132.2067ms iteration = 3 status = {"generation":2}
31832025-08-26T21:48:01.802ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
31842025-08-26T21:48:01.802ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
31852025-08-26T21:48:01.802ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 127.026074ms iteration = 3 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
31862025-08-26T21:48:01.844ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 153.186254ms iteration = 5 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
31872025-08-26T21:48:01.861ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
31882025-08-26T21:48:01.861ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
31892025-08-26T21:48:01.861ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "edc63dac-0cf9-4945-bcd6-280df1a52f92", "content-length": "26", "date": "Tue, 26 Aug 2025 21:48:01 GMT"} })
31902025-08-26T21:48:01.861ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
31912025-08-26T21:48:01.863ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
31922025-08-26T21:48:01.863ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 187.812293ms iteration = 5 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
31932025-08-26T21:48:01.863ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
31942025-08-26T21:48:01.863ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
31952025-08-26T21:48:01.863ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 175.488396ms iteration = 5 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
31962025-08-26T21:48:01.864ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
31972025-08-26T21:48:01.870ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
31982025-08-26T21:48:01.894ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
31992025-08-26T21:48:01.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
32002025-08-26T21:48:01.897ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
32012025-08-26T21:48:01.897ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32022025-08-26T21:48:01.898ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c7c4422-100a-4907-8851-f132c7a28b4b", "content-length": "26", "date": "Tue, 26 Aug 2025 21:48:01 GMT"} })
32032025-08-26T21:48:01.898ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32042025-08-26T21:48:01.899ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
32052025-08-26T21:48:01.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
32062025-08-26T21:48:01.899ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
32072025-08-26T21:48:01.901ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32082025-08-26T21:48:01.904ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 229.110495ms iteration = 6 status = {"physical_disks_added":0}
32092025-08-26T21:48:01.905ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
32102025-08-26T21:48:01.905ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
32112025-08-26T21:48:01.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
32122025-08-26T21:48:01.907ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32132025-08-26T21:48:01.909ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
32142025-08-26T21:48:01.909ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
32152025-08-26T21:48:01.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
32162025-08-26T21:48:01.910ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
32172025-08-26T21:48:01.910ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32182025-08-26T21:48:01.910ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a2ac11f-1b45-478e-aa29-70c43c8fd516", "content-length": "26", "date": "Tue, 26 Aug 2025 21:48:01 GMT"} })
32192025-08-26T21:48:01.910ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32202025-08-26T21:48:01.912ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
32212025-08-26T21:48:01.958ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
32222025-08-26T21:48:01.958ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
32232025-08-26T21:48:01.959ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "77fac660-3660-47c0-85c2-ad86de7734d7", "content-length": "372", "date": "Tue, 26 Aug 2025 21:48:01 GMT"} })
32242025-08-26T21:48:01.959ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32252025-08-26T21:48:01.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32262025-08-26T21:48:01.987ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
32272025-08-26T21:48:01.987ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
32282025-08-26T21:48:01.988ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "343fdd8e-ac8f-4f19-8609-578276a565ad", "date": "Tue, 26 Aug 2025 21:48:01 GMT"} })
32292025-08-26T21:48:01.988ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32302025-08-26T21:48:01.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32312025-08-26T21:48:01.990ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32322025-08-26T21:48:01.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32332025-08-26T21:48:01.994ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32342025-08-26T21:48:01.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32352025-08-26T21:48:01.998ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
32362025-08-26T21:48:01.998ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
32372025-08-26T21:48:01.998ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "cf7ebd80-d7ef-4ecf-946f-c70395ccf7cc", "date": "Tue, 26 Aug 2025 21:48:01 GMT"} })
32382025-08-26T21:48:01.999ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 308.468643ms iteration = 5 status = {}
32392025-08-26T21:48:02.009ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
32402025-08-26T21:48:02.058ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
32412025-08-26T21:48:02.059ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1b89a75-ab7a-4106-b6f2-6845a06b5041", "content-length": "691", "date": "Tue, 26 Aug 2025 21:48:01 GMT"} })
32422025-08-26T21:48:02.059ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
32432025-08-26T21:48:02.059ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
32442025-08-26T21:48:02.059ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
32452025-08-26T21:48:02.060ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
32462025-08-26T21:48:02.060ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 372.131109ms iteration = 3 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
32472025-08-26T21:48:02.061ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 385.593423ms iteration = 5 status = {}
32482025-08-26T21:48:02.074ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 408.235332ms iteration = 5 status = {}
32492025-08-26T21:48:02.095ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
32502025-08-26T21:48:02.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
32512025-08-26T21:48:02.097ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 431.968968ms iteration = 5 status = null
32522025-08-26T21:48:02.131ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
32532025-08-26T21:48:02.132ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "37f84ee6-f735-467b-8a38-f122ddaa3aad", "date": "Tue, 26 Aug 2025 21:48:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
32542025-08-26T21:48:02.132ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
32552025-08-26T21:48:02.134ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "69a34c10-ec00-4130-ad7a-fa86a849c29a", "date": "Tue, 26 Aug 2025 21:48:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
32562025-08-26T21:48:02.134ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
32572025-08-26T21:48:02.135ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
32582025-08-26T21:48:02.135ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 470.041456ms iteration = 5 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
32592025-08-26T21:48:02.141ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
32602025-08-26T21:48:02.266ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
32612025-08-26T21:48:02.267ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "327fd59e-2627-4900-b80e-de11e6325d9f", "date": "Tue, 26 Aug 2025 21:48:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
32622025-08-26T21:48:02.267ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
32632025-08-26T21:48:02.268ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "233bbd85-9274-42cd-a685-930f03183be0", "date": "Tue, 26 Aug 2025 21:48:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
32642025-08-26T21:48:02.268ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
32652025-08-26T21:48:02.269ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "d4574431-347d-4b58-a483-f2ba78d862cc", "date": "Tue, 26 Aug 2025 21:48:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
32662025-08-26T21:48:02.269ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
32672025-08-26T21:48:02.271ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
32682025-08-26T21:48:02.271ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 605.96782ms iteration = 5 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
32692025-08-26T21:48:02.293ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32702025-08-26T21:48:02.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32712025-08-26T21:48:02.296ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
32722025-08-26T21:48:02.296ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32732025-08-26T21:48:02.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32742025-08-26T21:48:02.307ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
32752025-08-26T21:48:02.307ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
32762025-08-26T21:48:02.308ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
32772025-08-26T21:48:02.308ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
32782025-08-26T21:48:02.308ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
32792025-08-26T21:48:02.308ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
32802025-08-26T21:48:02.308ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
32812025-08-26T21:48:02.308ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
32822025-08-26T21:48:02.308ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
32832025-08-26T21:48:02.308ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
32842025-08-26T21:48:02.308ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
32852025-08-26T21:48:02.308ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32862025-08-26T21:48:02.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32872025-08-26T21:48:02.312ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
32882025-08-26T21:48:02.312ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
32892025-08-26T21:48:02.405ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32902025-08-26T21:48:02.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32912025-08-26T21:48:02.408ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32922025-08-26T21:48:02.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32932025-08-26T21:48:02.410ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32942025-08-26T21:48:02.413ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
32952025-08-26T21:48:02.413ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32962025-08-26T21:48:02.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32972025-08-26T21:48:02.413ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "9e12c1ae-c06f-4026-a9a9-be6853f02c07", "date": "Tue, 26 Aug 2025 21:48:01 GMT"} })
32982025-08-26T21:48:02.413ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
32992025-08-26T21:48:02.413ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
33002025-08-26T21:48:02.414ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "5dcedbd1-7e45-4379-9921-4ce22ef51e97", "content-length": "682", "date": "Tue, 26 Aug 2025 21:48:01 GMT"} })
33012025-08-26T21:48:02.414ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33022025-08-26T21:48:02.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33032025-08-26T21:48:02.415ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 739.766466ms iteration = 5 status = 0
33042025-08-26T21:48:02.431ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33052025-08-26T21:48:02.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33062025-08-26T21:48:02.435ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33072025-08-26T21:48:02.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33082025-08-26T21:48:02.437ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33092025-08-26T21:48:02.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33102025-08-26T21:48:02.450ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 775.218205ms iteration = 5 status = {}
33112025-08-26T21:48:05.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 334 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 4efdf591-5034-4d4f-a6ed-4dc133b8421b response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
33122025-08-26T21:48:06.668ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 26 reason = Timeout
33132025-08-26T21:48:06.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
33142025-08-26T21:48:06.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
33152025-08-26T21:48:06.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33162025-08-26T21:48:06.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
33172025-08-26T21:48:06.673ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
33182025-08-26T21:48:06.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
33192025-08-26T21:48:06.675ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
33202025-08-26T21:48:06.675ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.834146ms iteration = 26 status = {"chicken_switches_updated":false}
33212025-08-26T21:48:11.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 27 reason = Timeout
33222025-08-26T21:48:11.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
33232025-08-26T21:48:11.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
33242025-08-26T21:48:11.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33252025-08-26T21:48:11.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
33262025-08-26T21:48:11.673ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
33272025-08-26T21:48:11.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
33282025-08-26T21:48:11.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 14 reason = Timeout
33292025-08-26T21:48:11.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33302025-08-26T21:48:11.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33312025-08-26T21:48:11.676ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
33322025-08-26T21:48:11.676ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.501392ms iteration = 27 status = {"chicken_switches_updated":false}
33332025-08-26T21:48:11.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33342025-08-26T21:48:11.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
33352025-08-26T21:48:11.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33362025-08-26T21:48:11.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33372025-08-26T21:48:11.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33382025-08-26T21:48:11.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33392025-08-26T21:48:11.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33402025-08-26T21:48:11.687ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
33412025-08-26T21:48:11.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33422025-08-26T21:48:11.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33432025-08-26T21:48:11.688ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 14 reason = Timeout
33442025-08-26T21:48:11.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
33452025-08-26T21:48:11.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
33462025-08-26T21:48:11.689ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
33472025-08-26T21:48:11.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
33482025-08-26T21:48:11.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 2.914805ms iteration = 14 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
33492025-08-26T21:48:11.703ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
33502025-08-26T21:48:11.703ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.92249ms iteration = 14 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
33512025-08-26T21:48:15.346ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56248 req_id = c0603808-6c55-4dd0-93f7-d2eef96cfec2 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
33522025-08-26T21:48:15.347ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56248 req_id = c0603808-6c55-4dd0-93f7-d2eef96cfec2 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
33532025-08-26T21:48:15.350ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 4097 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56248 req_id = c0603808-6c55-4dd0-93f7-d2eef96cfec2 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
33542025-08-26T21:48:15.351ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56248 req_id = 32fdfb6d-0e67-41c0-9058-a0d6e4005cd9 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
33552025-08-26T21:48:15.351ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56248 req_id = 32fdfb6d-0e67-41c0-9058-a0d6e4005cd9 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
33562025-08-26T21:48:15.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 310 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 001026a5-358b-48aa-a055-a7702a01ff51 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
33572025-08-26T21:48:15.353ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2567 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:56248 req_id = 32fdfb6d-0e67-41c0-9058-a0d6e4005cd9 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
33582025-08-26T21:48:16.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 28 reason = Timeout
33592025-08-26T21:48:16.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
33602025-08-26T21:48:16.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
33612025-08-26T21:48:16.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33622025-08-26T21:48:16.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
33632025-08-26T21:48:16.673ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
33642025-08-26T21:48:16.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
33652025-08-26T21:48:16.675ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
33662025-08-26T21:48:16.675ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.759455ms iteration = 28 status = {"chicken_switches_updated":false}
33672025-08-26T21:48:21.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 29 reason = Timeout
33682025-08-26T21:48:21.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
33692025-08-26T21:48:21.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
33702025-08-26T21:48:21.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33712025-08-26T21:48:21.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
33722025-08-26T21:48:21.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
33732025-08-26T21:48:21.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
33742025-08-26T21:48:21.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
33752025-08-26T21:48:21.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.890275ms iteration = 29 status = {"chicken_switches_updated":false}
33762025-08-26T21:48:21.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 15 reason = Timeout
33772025-08-26T21:48:21.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33782025-08-26T21:48:21.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33792025-08-26T21:48:21.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33802025-08-26T21:48:21.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
33812025-08-26T21:48:21.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33822025-08-26T21:48:21.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33832025-08-26T21:48:21.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33842025-08-26T21:48:21.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33852025-08-26T21:48:21.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33862025-08-26T21:48:21.687ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
33872025-08-26T21:48:21.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33882025-08-26T21:48:21.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
33892025-08-26T21:48:21.687ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 15 reason = Timeout
33902025-08-26T21:48:21.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
33912025-08-26T21:48:21.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
33922025-08-26T21:48:21.689ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
33932025-08-26T21:48:21.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
33942025-08-26T21:48:21.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.119791ms iteration = 15 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
33952025-08-26T21:48:21.704ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
33962025-08-26T21:48:21.704ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.664156ms iteration = 15 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
33972025-08-26T21:48:25.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 354 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 2a18f37f-adc4-4511-88d2-48bd2d16d241 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
33982025-08-26T21:48:26.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 30 reason = Timeout
33992025-08-26T21:48:26.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
34002025-08-26T21:48:26.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
34012025-08-26T21:48:26.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34022025-08-26T21:48:26.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
34032025-08-26T21:48:26.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
34042025-08-26T21:48:26.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
34052025-08-26T21:48:26.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
34062025-08-26T21:48:26.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.10649ms iteration = 30 status = {"chicken_switches_updated":false}
34072025-08-26T21:48:31.665ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 6 reason = Timeout
34082025-08-26T21:48:31.665ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 6 reason = Timeout
34092025-08-26T21:48:31.665ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 6 reason = Timeout
34102025-08-26T21:48:31.665ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
34112025-08-26T21:48:31.665ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
34122025-08-26T21:48:31.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
34132025-08-26T21:48:31.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
34142025-08-26T21:48:31.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 31 reason = Timeout
34152025-08-26T21:48:31.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
34162025-08-26T21:48:31.668ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1015, tv_nsec: 525518599 } })
34172025-08-26T21:48:31.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
34182025-08-26T21:48:31.668ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
34192025-08-26T21:48:31.668ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1015, tv_nsec: 525916968 } })
34202025-08-26T21:48:31.668ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
34212025-08-26T21:48:31.669ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.401857ms iteration = 6 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
34222025-08-26T21:48:31.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34232025-08-26T21:48:31.676ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 6 reason = Timeout
34242025-08-26T21:48:31.676ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 6 reason = Timeout
34252025-08-26T21:48:31.676ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
34262025-08-26T21:48:31.676ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
34272025-08-26T21:48:31.676ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 6 reason = Timeout
34282025-08-26T21:48:31.676ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 6 reason = Timeout
34292025-08-26T21:48:31.676ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34302025-08-26T21:48:31.676ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
34312025-08-26T21:48:31.676ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 16 reason = Timeout
34322025-08-26T21:48:31.676ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 6 reason = Timeout
34332025-08-26T21:48:31.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34342025-08-26T21:48:31.676ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 7 reason = Timeout
34352025-08-26T21:48:31.676ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
34362025-08-26T21:48:31.676ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34372025-08-26T21:48:31.676ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
34382025-08-26T21:48:31.676ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
34392025-08-26T21:48:31.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
34402025-08-26T21:48:31.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
34412025-08-26T21:48:31.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
34422025-08-26T21:48:31.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
34432025-08-26T21:48:31.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34442025-08-26T21:48:31.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
34452025-08-26T21:48:31.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34462025-08-26T21:48:31.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
34472025-08-26T21:48:31.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34482025-08-26T21:48:31.678ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34492025-08-26T21:48:31.678ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34502025-08-26T21:48:31.678ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34512025-08-26T21:48:31.679ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34522025-08-26T21:48:31.681ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
34532025-08-26T21:48:31.681ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 13.833422ms iteration = 31 status = {"chicken_switches_updated":false}
34542025-08-26T21:48:31.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
34552025-08-26T21:48:31.683ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34562025-08-26T21:48:31.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
34572025-08-26T21:48:31.683ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
34582025-08-26T21:48:31.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34592025-08-26T21:48:31.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
34602025-08-26T21:48:31.683ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
34612025-08-26T21:48:31.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
34622025-08-26T21:48:31.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
34632025-08-26T21:48:31.684ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
34642025-08-26T21:48:31.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
34652025-08-26T21:48:31.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34662025-08-26T21:48:31.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
34672025-08-26T21:48:31.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34682025-08-26T21:48:31.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
34692025-08-26T21:48:31.684ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
34702025-08-26T21:48:31.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
34712025-08-26T21:48:31.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
34722025-08-26T21:48:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
34732025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
34742025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 6 reason = Timeout
34752025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
34762025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
34772025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 6 reason = Timeout
34782025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
34792025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
34802025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
34812025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 16 reason = Timeout
34822025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34832025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
34842025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 6 reason = Timeout
34852025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 8.79\u{b5}s iteration = 6 status = {"disabled":true,"errors":[],"sps":[]}
34862025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 6 reason = Timeout
34872025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
34882025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34892025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 6 reason = Timeout
34902025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
34912025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
34922025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 6 reason = Timeout
34932025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
34942025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
34952025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 6 reason = Timeout
34962025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
34972025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
34982025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 6 reason = Timeout
34992025-08-26T21:48:31.689ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
35002025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
35012025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
35022025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 6 reason = Timeout
35032025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
35042025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
35052025-08-26T21:48:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35062025-08-26T21:48:31.690ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35072025-08-26T21:48:31.690ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
35082025-08-26T21:48:31.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
35092025-08-26T21:48:31.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 14.801334ms iteration = 6 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
35102025-08-26T21:48:31.691ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
35112025-08-26T21:48:31.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35122025-08-26T21:48:31.691ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1015, tv_nsec: 549223494 } })
35132025-08-26T21:48:31.691ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35142025-08-26T21:48:31.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35152025-08-26T21:48:31.692ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
35162025-08-26T21:48:31.693ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
35172025-08-26T21:48:31.693ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
35182025-08-26T21:48:31.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
35192025-08-26T21:48:31.693ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
35202025-08-26T21:48:31.693ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
35212025-08-26T21:48:31.693ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
35222025-08-26T21:48:31.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
35232025-08-26T21:48:31.696ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35242025-08-26T21:48:31.696ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35252025-08-26T21:48:31.696ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 8.212965ms iteration = 16 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
35262025-08-26T21:48:31.696ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 7.427027ms iteration = 6 status = {"errors":[],"finish_invoked_ok":[]}
35272025-08-26T21:48:31.696ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35282025-08-26T21:48:31.696ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35292025-08-26T21:48:31.699ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
35302025-08-26T21:48:31.699ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
35312025-08-26T21:48:31.699ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
35322025-08-26T21:48:31.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
35332025-08-26T21:48:31.700ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
35342025-08-26T21:48:31.700ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
35352025-08-26T21:48:31.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
35362025-08-26T21:48:31.701ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
35372025-08-26T21:48:31.701ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
35382025-08-26T21:48:31.701ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
35392025-08-26T21:48:31.701ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
35402025-08-26T21:48:31.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35412025-08-26T21:48:31.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35422025-08-26T21:48:31.703ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
35432025-08-26T21:48:31.703ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35442025-08-26T21:48:31.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
35452025-08-26T21:48:31.703ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35462025-08-26T21:48:31.706ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 30.062888ms iteration = 7 status = {"physical_disks_added":0}
35472025-08-26T21:48:31.706ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
35482025-08-26T21:48:31.706ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
35492025-08-26T21:48:31.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
35502025-08-26T21:48:31.707ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
35512025-08-26T21:48:31.707ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 20.115924ms iteration = 6 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
35522025-08-26T21:48:31.740ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
35532025-08-26T21:48:31.740ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 64.256878ms iteration = 16 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
35542025-08-26T21:48:31.755ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 66.595382ms iteration = 6 status = {"errors":[],"garbage_collect_requested":[]}
35552025-08-26T21:48:31.760ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
35562025-08-26T21:48:31.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
35572025-08-26T21:48:31.761ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 72.712329ms iteration = 6 status = {"errors":[],"requests_created_ok":[]}
35582025-08-26T21:48:31.764ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
35592025-08-26T21:48:31.764ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
35602025-08-26T21:48:31.764ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "bee3fce1-c0fc-4d67-9ea8-ed13fa1d2c37", "content-length": "26", "date": "Tue, 26 Aug 2025 21:48:31 GMT"} })
35612025-08-26T21:48:31.764ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
35622025-08-26T21:48:31.768ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
35632025-08-26T21:48:31.768ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
35642025-08-26T21:48:31.768ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 80.173098ms iteration = 6 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
35652025-08-26T21:48:31.768ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
35662025-08-26T21:48:31.768ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
35672025-08-26T21:48:31.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
35682025-08-26T21:48:31.774ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
35692025-08-26T21:48:31.774ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 97.994159ms iteration = 6 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
35702025-08-26T21:48:31.783ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
35712025-08-26T21:48:31.783ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
35722025-08-26T21:48:31.784ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1470f164-ad5c-432a-ab7e-204bedc93b12", "content-length": "26", "date": "Tue, 26 Aug 2025 21:48:31 GMT"} })
35732025-08-26T21:48:31.784ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
35742025-08-26T21:48:31.785ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
35752025-08-26T21:48:31.803ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
35762025-08-26T21:48:31.803ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
35772025-08-26T21:48:31.803ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff990352-5275-482b-964e-4cee07e7d1d8", "content-length": "26", "date": "Tue, 26 Aug 2025 21:48:31 GMT"} })
35782025-08-26T21:48:31.804ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
35792025-08-26T21:48:31.805ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
35802025-08-26T21:48:31.805ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
35812025-08-26T21:48:31.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
35822025-08-26T21:48:31.808ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 119.508634ms iteration = 6 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
35832025-08-26T21:48:31.815ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 126.769718ms iteration = 6 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
35842025-08-26T21:48:31.852ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
35852025-08-26T21:48:31.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
35862025-08-26T21:48:31.897ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
35872025-08-26T21:48:31.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
35882025-08-26T21:48:31.899ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35892025-08-26T21:48:31.902ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
35902025-08-26T21:48:31.902ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
35912025-08-26T21:48:31.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
35922025-08-26T21:48:31.905ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35932025-08-26T21:48:31.907ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
35942025-08-26T21:48:31.907ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
35952025-08-26T21:48:31.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
35962025-08-26T21:48:32.054ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
35972025-08-26T21:48:32.055ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4462e62-3717-44f0-a7fe-db3d38081c90", "content-length": "691", "date": "Tue, 26 Aug 2025 21:48:31 GMT"} })
35982025-08-26T21:48:32.055ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
35992025-08-26T21:48:32.055ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
36002025-08-26T21:48:32.055ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
36012025-08-26T21:48:32.056ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 380.721743ms iteration = 6 status = {}
36022025-08-26T21:48:32.057ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36032025-08-26T21:48:32.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36042025-08-26T21:48:32.059ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36052025-08-26T21:48:32.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36062025-08-26T21:48:32.061ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36072025-08-26T21:48:32.064ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
36082025-08-26T21:48:32.064ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
36092025-08-26T21:48:32.064ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
36102025-08-26T21:48:32.066ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 390.393531ms iteration = 6 status = 0
36112025-08-26T21:48:32.071ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 405.42983ms iteration = 6 status = {}
36122025-08-26T21:48:32.116ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
36132025-08-26T21:48:32.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
36142025-08-26T21:48:32.119ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 453.666437ms iteration = 6 status = null
36152025-08-26T21:48:32.121ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
36162025-08-26T21:48:32.121ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
36172025-08-26T21:48:32.121ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "0cae9092-68bd-4dac-9134-b5fa6e8e0d27", "content-length": "372", "date": "Tue, 26 Aug 2025 21:48:31 GMT"} })
36182025-08-26T21:48:32.121ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36192025-08-26T21:48:32.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36202025-08-26T21:48:32.127ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36212025-08-26T21:48:32.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36222025-08-26T21:48:32.130ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
36232025-08-26T21:48:32.130ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36242025-08-26T21:48:32.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36252025-08-26T21:48:32.138ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
36262025-08-26T21:48:32.138ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
36272025-08-26T21:48:32.138ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
36282025-08-26T21:48:32.138ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
36292025-08-26T21:48:32.138ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
36302025-08-26T21:48:32.138ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
36312025-08-26T21:48:32.138ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
36322025-08-26T21:48:32.138ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
36332025-08-26T21:48:32.139ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
36342025-08-26T21:48:32.139ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
36352025-08-26T21:48:32.139ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
36362025-08-26T21:48:32.139ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36372025-08-26T21:48:32.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36382025-08-26T21:48:32.142ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
36392025-08-26T21:48:32.142ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
36402025-08-26T21:48:32.174ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
36412025-08-26T21:48:32.174ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
36422025-08-26T21:48:32.174ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "337fb1a6-99cc-47e3-8d25-181784f1f261", "date": "Tue, 26 Aug 2025 21:48:31 GMT"} })
36432025-08-26T21:48:32.174ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36442025-08-26T21:48:32.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36452025-08-26T21:48:32.177ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36462025-08-26T21:48:32.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36472025-08-26T21:48:32.180ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
36482025-08-26T21:48:32.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
36492025-08-26T21:48:32.183ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
36502025-08-26T21:48:32.183ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
36512025-08-26T21:48:32.185ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "b8df84ed-89a9-4c92-a4db-686c9861b8f4", "date": "Tue, 26 Aug 2025 21:48:31 GMT"} })
36522025-08-26T21:48:32.186ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 497.351301ms iteration = 6 status = {}
36532025-08-26T21:48:32.256ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "8b84e0ff-0af7-4cdb-8eaa-56c32c1ba5ed", "date": "Tue, 26 Aug 2025 21:48:31 GMT"} })
36542025-08-26T21:48:32.256ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
36552025-08-26T21:48:32.256ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
36562025-08-26T21:48:32.256ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "803a2f03-8b53-46be-ba94-e0db5c2f8cb3", "content-length": "682", "date": "Tue, 26 Aug 2025 21:48:31 GMT"} })
36572025-08-26T21:48:32.256ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36582025-08-26T21:48:32.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36592025-08-26T21:48:32.259ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36602025-08-26T21:48:32.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36612025-08-26T21:48:32.263ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36622025-08-26T21:48:32.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36632025-08-26T21:48:32.265ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
36642025-08-26T21:48:32.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
36652025-08-26T21:48:32.272ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 596.21716ms iteration = 6 status = {}
36662025-08-26T21:48:35.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 375 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 1295307b-c55e-4862-a80b-79b98784b97c response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
36672025-08-26T21:48:36.668ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 32 reason = Timeout
36682025-08-26T21:48:36.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
36692025-08-26T21:48:36.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
36702025-08-26T21:48:36.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36712025-08-26T21:48:36.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
36722025-08-26T21:48:36.673ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
36732025-08-26T21:48:36.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
36742025-08-26T21:48:36.675ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
36752025-08-26T21:48:36.675ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.491058ms iteration = 32 status = {"chicken_switches_updated":false}
36762025-08-26T21:48:41.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 33 reason = Timeout
36772025-08-26T21:48:41.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
36782025-08-26T21:48:41.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
36792025-08-26T21:48:41.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36802025-08-26T21:48:41.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
36812025-08-26T21:48:41.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
36822025-08-26T21:48:41.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
36832025-08-26T21:48:41.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
36842025-08-26T21:48:41.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.901975ms iteration = 33 status = {"chicken_switches_updated":false}
36852025-08-26T21:48:41.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 17 reason = Timeout
36862025-08-26T21:48:41.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36872025-08-26T21:48:41.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36882025-08-26T21:48:41.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36892025-08-26T21:48:41.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
36902025-08-26T21:48:41.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36912025-08-26T21:48:41.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36922025-08-26T21:48:41.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36932025-08-26T21:48:41.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
36942025-08-26T21:48:41.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36952025-08-26T21:48:41.688ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 17 reason = Timeout
36962025-08-26T21:48:41.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
36972025-08-26T21:48:41.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
36982025-08-26T21:48:41.689ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
36992025-08-26T21:48:41.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
37002025-08-26T21:48:41.690ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
37012025-08-26T21:48:41.690ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37022025-08-26T21:48:41.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37032025-08-26T21:48:41.691ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.743714ms iteration = 17 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
37042025-08-26T21:48:41.708ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
37052025-08-26T21:48:41.708ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 32.632665ms iteration = 17 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
37062025-08-26T21:48:45.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 366 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 1d06deed-7f61-44e0-aa73-99c3fc40cc74 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
37072025-08-26T21:48:46.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 34 reason = Timeout
37082025-08-26T21:48:46.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
37092025-08-26T21:48:46.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
37102025-08-26T21:48:46.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37112025-08-26T21:48:46.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
37122025-08-26T21:48:46.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
37132025-08-26T21:48:46.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
37142025-08-26T21:48:46.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
37152025-08-26T21:48:46.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.314414ms iteration = 34 status = {"chicken_switches_updated":false}
37162025-08-26T21:48:51.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 35 reason = Timeout
37172025-08-26T21:48:51.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
37182025-08-26T21:48:51.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
37192025-08-26T21:48:51.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37202025-08-26T21:48:51.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
37212025-08-26T21:48:51.673ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
37222025-08-26T21:48:51.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
37232025-08-26T21:48:51.675ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
37242025-08-26T21:48:51.675ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.258164ms iteration = 35 status = {"chicken_switches_updated":false}
37252025-08-26T21:48:51.676ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 18 reason = Timeout
37262025-08-26T21:48:51.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37272025-08-26T21:48:51.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37282025-08-26T21:48:51.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37292025-08-26T21:48:51.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
37302025-08-26T21:48:51.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37312025-08-26T21:48:51.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37322025-08-26T21:48:51.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37332025-08-26T21:48:51.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37342025-08-26T21:48:51.687ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 18 reason = Timeout
37352025-08-26T21:48:51.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
37362025-08-26T21:48:51.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37372025-08-26T21:48:51.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
37382025-08-26T21:48:51.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
37392025-08-26T21:48:51.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
37402025-08-26T21:48:51.690ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
37412025-08-26T21:48:51.690ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37422025-08-26T21:48:51.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
37432025-08-26T21:48:51.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.305015ms iteration = 18 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
37442025-08-26T21:48:51.702ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
37452025-08-26T21:48:51.702ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.019766ms iteration = 18 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
37462025-08-26T21:48:55.354ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 588 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = b650828c-1eda-4942-b72f-9f1cf77b5614 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
37472025-08-26T21:48:56.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 36 reason = Timeout
37482025-08-26T21:48:56.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
37492025-08-26T21:48:56.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
37502025-08-26T21:48:56.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37512025-08-26T21:48:56.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
37522025-08-26T21:48:56.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
37532025-08-26T21:48:56.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
37542025-08-26T21:48:56.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
37552025-08-26T21:48:56.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.241003ms iteration = 36 status = {"chicken_switches_updated":false}
37562025-08-26T21:49:00.345ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:46227
37572025-08-26T21:49:00.346ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46227 req_id = c734947f-c8d6-45d6-8e61-df22ed47a1a8 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
37582025-08-26T21:49:00.346ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46227 req_id = c734947f-c8d6-45d6-8e61-df22ed47a1a8 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
37592025-08-26T21:49:00.349ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3268 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46227 req_id = c734947f-c8d6-45d6-8e61-df22ed47a1a8 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
37602025-08-26T21:49:00.349ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46227 req_id = 0075763f-6fd2-493b-ae40-68d39c823a28 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
37612025-08-26T21:49:00.349ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46227 req_id = 0075763f-6fd2-493b-ae40-68d39c823a28 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
37622025-08-26T21:49:00.352ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2726 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46227 req_id = 0075763f-6fd2-493b-ae40-68d39c823a28 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
37632025-08-26T21:49:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 4 reason = Timeout
37642025-08-26T21:49:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 4 reason = Timeout
37652025-08-26T21:49:01.665ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
37662025-08-26T21:49:01.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
37672025-08-26T21:49:01.665ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 4 reason = Timeout
37682025-08-26T21:49:01.665ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:39:01.665402745 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
37692025-08-26T21:49:01.665ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
37702025-08-26T21:49:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 4 reason = Timeout
37712025-08-26T21:49:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 4 reason = Timeout
37722025-08-26T21:49:01.665ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
37732025-08-26T21:49:01.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
37742025-08-26T21:49:01.665ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 4 reason = Timeout
37752025-08-26T21:49:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 6 reason = Timeout
37762025-08-26T21:49:01.665ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 7 reason = Timeout
37772025-08-26T21:49:01.665ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
37782025-08-26T21:49:01.665ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
37792025-08-26T21:49:01.665ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 7 reason = Timeout
37802025-08-26T21:49:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 6 reason = Timeout
37812025-08-26T21:49:01.665ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 7 reason = Timeout
37822025-08-26T21:49:01.665ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
37832025-08-26T21:49:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
37842025-08-26T21:49:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
37852025-08-26T21:49:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
37862025-08-26T21:49:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
37872025-08-26T21:49:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
37882025-08-26T21:49:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
37892025-08-26T21:49:01.666ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1045, tv_nsec: 523531615 } })
37902025-08-26T21:49:01.666ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1045, tv_nsec: 523530175 } })
37912025-08-26T21:49:01.666ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 5 reason = Timeout
37922025-08-26T21:49:01.666ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
37932025-08-26T21:49:01.666ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.54\u{b5}s iteration = 5 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
37942025-08-26T21:49:01.666ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
37952025-08-26T21:49:01.666ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.139416ms iteration = 4 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
37962025-08-26T21:49:01.666ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
37972025-08-26T21:49:01.666ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
37982025-08-26T21:49:01.666ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
37992025-08-26T21:49:01.666ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.211397ms iteration = 4 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
38002025-08-26T21:49:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
38012025-08-26T21:49:01.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
38022025-08-26T21:49:01.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 37 reason = Timeout
38032025-08-26T21:49:01.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
38042025-08-26T21:49:01.667ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:39:01.665402745 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
38052025-08-26T21:49:01.667ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 2.486217ms iteration = 4 status = {"expiration":"2025-08-26T21:39:01.665402745Z","pruned":[]}
38062025-08-26T21:49:01.667ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38072025-08-26T21:49:01.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
38082025-08-26T21:49:01.668ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1045, tv_nsec: 526036931 } })
38092025-08-26T21:49:01.668ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
38102025-08-26T21:49:01.668ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.356156ms iteration = 7 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
38112025-08-26T21:49:01.668ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38122025-08-26T21:49:01.669ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38132025-08-26T21:49:01.669ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1045, tv_nsec: 526780308 } })
38142025-08-26T21:49:01.669ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
38152025-08-26T21:49:01.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38162025-08-26T21:49:01.670ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
38172025-08-26T21:49:01.670ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
38182025-08-26T21:49:01.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
38192025-08-26T21:49:01.672ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
38202025-08-26T21:49:01.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
38212025-08-26T21:49:01.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
38222025-08-26T21:49:01.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
38232025-08-26T21:49:01.673ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
38242025-08-26T21:49:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
38252025-08-26T21:49:01.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 6 reason = Timeout
38262025-08-26T21:49:01.674ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
38272025-08-26T21:49:01.674ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 13.69\u{b5}s iteration = 6 status = "Disabled"
38282025-08-26T21:49:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
38292025-08-26T21:49:01.674ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
38302025-08-26T21:49:01.674ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
38312025-08-26T21:49:01.674ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
38322025-08-26T21:49:01.674ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
38332025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38342025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 7 reason = Timeout
38352025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
38362025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 4 reason = Timeout
38372025-08-26T21:49:01.675ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
38382025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
38392025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
38402025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 7 reason = Timeout
38412025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
38422025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 8 reason = Timeout
38432025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
38442025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 3 reason = Timeout
38452025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 7 reason = Timeout
38462025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
38472025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
38482025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 19 reason = Timeout
38492025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38502025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 7 reason = Timeout
38512025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
38522025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 7 reason = Timeout
38532025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
38542025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
38552025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
38562025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
38572025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
38582025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
38592025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
38602025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
38612025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
38622025-08-26T21:49:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
38632025-08-26T21:49:01.676ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
38642025-08-26T21:49:01.676ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
38652025-08-26T21:49:01.676ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
38662025-08-26T21:49:01.676ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
38672025-08-26T21:49:01.676ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
38682025-08-26T21:49:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
38692025-08-26T21:49:01.676ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
38702025-08-26T21:49:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
38712025-08-26T21:49:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
38722025-08-26T21:49:01.677ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
38732025-08-26T21:49:01.677ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38742025-08-26T21:49:01.677ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38752025-08-26T21:49:01.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
38762025-08-26T21:49:01.677ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
38772025-08-26T21:49:01.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
38782025-08-26T21:49:01.678ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
38792025-08-26T21:49:01.678ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 10.20115ms iteration = 37 status = {"chicken_switches_updated":false}
38802025-08-26T21:49:01.678ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38812025-08-26T21:49:01.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38822025-08-26T21:49:01.678ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38832025-08-26T21:49:01.678ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38842025-08-26T21:49:01.679ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38852025-08-26T21:49:01.679ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38862025-08-26T21:49:01.679ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38872025-08-26T21:49:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
38882025-08-26T21:49:01.682ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
38892025-08-26T21:49:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
38902025-08-26T21:49:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
38912025-08-26T21:49:01.686ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
38922025-08-26T21:49:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
38932025-08-26T21:49:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
38942025-08-26T21:49:01.687ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
38952025-08-26T21:49:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
38962025-08-26T21:49:01.687ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
38972025-08-26T21:49:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
38982025-08-26T21:49:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
38992025-08-26T21:49:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
39002025-08-26T21:49:01.687ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
39012025-08-26T21:49:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
39022025-08-26T21:49:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
39032025-08-26T21:49:01.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39042025-08-26T21:49:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39052025-08-26T21:49:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
39062025-08-26T21:49:01.687ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
39072025-08-26T21:49:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
39082025-08-26T21:49:01.687ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
39092025-08-26T21:49:01.688ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 4 reason = Timeout
39102025-08-26T21:49:01.688ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
39112025-08-26T21:49:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
39122025-08-26T21:49:01.688ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
39132025-08-26T21:49:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
39142025-08-26T21:49:01.688ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39152025-08-26T21:49:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
39162025-08-26T21:49:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
39172025-08-26T21:49:01.688ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39182025-08-26T21:49:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
39192025-08-26T21:49:01.688ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
39202025-08-26T21:49:01.688ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 7 reason = Timeout
39212025-08-26T21:49:01.688ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
39222025-08-26T21:49:01.688ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
39232025-08-26T21:49:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39242025-08-26T21:49:01.688ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 19 reason = Timeout
39252025-08-26T21:49:01.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39262025-08-26T21:49:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
39272025-08-26T21:49:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
39282025-08-26T21:49:01.688ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 7 reason = Timeout
39292025-08-26T21:49:01.688ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
39302025-08-26T21:49:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39312025-08-26T21:49:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
39322025-08-26T21:49:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
39332025-08-26T21:49:01.688ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 4 reason = Timeout
39342025-08-26T21:49:01.688ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
39352025-08-26T21:49:01.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
39362025-08-26T21:49:01.689ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 4 reason = Timeout
39372025-08-26T21:49:01.689ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
39382025-08-26T21:49:01.689ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 7 reason = Timeout
39392025-08-26T21:49:01.689ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
39402025-08-26T21:49:01.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
39412025-08-26T21:49:01.689ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 7 reason = Timeout
39422025-08-26T21:49:01.689ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
39432025-08-26T21:49:01.690ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 4 reason = Timeout
39442025-08-26T21:49:01.690ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
39452025-08-26T21:49:01.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
39462025-08-26T21:49:01.690ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 4 reason = Timeout
39472025-08-26T21:49:01.690ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
39482025-08-26T21:49:01.690ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 7 reason = Timeout
39492025-08-26T21:49:01.690ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 7 reason = Timeout
39502025-08-26T21:49:01.690ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
39512025-08-26T21:49:01.690ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 12.941\u{b5}s iteration = 7 status = {"disabled":true,"errors":[],"sps":[]}
39522025-08-26T21:49:01.690ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 7 reason = Timeout
39532025-08-26T21:49:01.690ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
39542025-08-26T21:49:01.690ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 7 reason = Timeout
39552025-08-26T21:49:01.690ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
39562025-08-26T21:49:01.690ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 7 reason = Timeout
39572025-08-26T21:49:01.690ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
39582025-08-26T21:49:01.690ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39592025-08-26T21:49:01.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
39602025-08-26T21:49:01.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
39612025-08-26T21:49:01.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
39622025-08-26T21:49:01.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
39632025-08-26T21:49:01.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
39642025-08-26T21:49:01.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39652025-08-26T21:49:01.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
39662025-08-26T21:49:01.692ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39672025-08-26T21:49:01.693ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
39682025-08-26T21:49:01.693ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
39692025-08-26T21:49:01.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39702025-08-26T21:49:01.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
39712025-08-26T21:49:01.693ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39722025-08-26T21:49:01.694ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 5.553115ms iteration = 4 status = {"errors":[],"found_port_ok":[]}
39732025-08-26T21:49:01.695ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
39742025-08-26T21:49:01.695ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
39752025-08-26T21:49:01.695ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 20.243317ms iteration = 4 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
39762025-08-26T21:49:01.695ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
39772025-08-26T21:49:01.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
39782025-08-26T21:49:01.696ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39792025-08-26T21:49:01.697ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
39802025-08-26T21:49:01.697ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
39812025-08-26T21:49:01.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
39822025-08-26T21:49:01.697ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39832025-08-26T21:49:01.698ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 8.671546ms iteration = 7 status = {"errors":[],"garbage_collect_requested":[]}
39842025-08-26T21:49:01.698ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1045, tv_nsec: 556384685 } })
39852025-08-26T21:49:01.699ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
39862025-08-26T21:49:01.699ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
39872025-08-26T21:49:01.699ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
39882025-08-26T21:49:01.699ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
39892025-08-26T21:49:01.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
39902025-08-26T21:49:01.700ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39912025-08-26T21:49:01.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
39922025-08-26T21:49:01.700ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
39932025-08-26T21:49:01.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
39942025-08-26T21:49:01.701ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
39952025-08-26T21:49:01.701ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
39962025-08-26T21:49:01.701ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
39972025-08-26T21:49:01.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
39982025-08-26T21:49:01.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
39992025-08-26T21:49:01.701ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40002025-08-26T21:49:01.701ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
40012025-08-26T21:49:01.701ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 13.849653ms iteration = 19 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
40022025-08-26T21:49:01.702ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
40032025-08-26T21:49:01.702ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
40042025-08-26T21:49:01.702ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 12.252396ms iteration = 4 status = {"by_rx":{},"error":null}
40052025-08-26T21:49:01.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
40062025-08-26T21:49:01.702ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40072025-08-26T21:49:01.702ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40082025-08-26T21:49:01.704ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
40092025-08-26T21:49:01.704ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
40102025-08-26T21:49:01.704ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 15.065109ms iteration = 4 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
40112025-08-26T21:49:01.708ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
40122025-08-26T21:49:01.708ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40132025-08-26T21:49:01.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40142025-08-26T21:49:01.708ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
40152025-08-26T21:49:01.708ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
40162025-08-26T21:49:01.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
40172025-08-26T21:49:01.708ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40182025-08-26T21:49:01.710ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
40192025-08-26T21:49:01.710ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
40202025-08-26T21:49:01.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
40212025-08-26T21:49:01.711ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
40222025-08-26T21:49:01.711ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
40232025-08-26T21:49:01.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
40242025-08-26T21:49:01.712ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40252025-08-26T21:49:01.712ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40262025-08-26T21:49:01.717ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
40272025-08-26T21:49:01.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
40282025-08-26T21:49:01.717ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
40292025-08-26T21:49:01.717ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40302025-08-26T21:49:01.717ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
40312025-08-26T21:49:01.717ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 28.594065ms iteration = 7 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
40322025-08-26T21:49:01.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40332025-08-26T21:49:01.718ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
40342025-08-26T21:49:01.718ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
40352025-08-26T21:49:01.718ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40362025-08-26T21:49:01.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
40372025-08-26T21:49:01.720ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
40382025-08-26T21:49:01.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
40392025-08-26T21:49:01.721ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
40402025-08-26T21:49:01.721ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40412025-08-26T21:49:01.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
40422025-08-26T21:49:01.722ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
40432025-08-26T21:49:01.722ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
40442025-08-26T21:49:01.722ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-08-26 21:46:04.065217 UTC dns_group = internal generation = 1
40452025-08-26T21:49:01.722ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 56.75091ms iteration = 4 status = {"generation":1}
40462025-08-26T21:49:01.722ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 47.067151ms iteration = 3 status = {"nsuccess":5}
40472025-08-26T21:49:01.734ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 44.429872ms iteration = 4 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
40482025-08-26T21:49:01.751ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
40492025-08-26T21:49:01.751ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 76.437992ms iteration = 19 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
40502025-08-26T21:49:01.778ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
40512025-08-26T21:49:01.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
40522025-08-26T21:49:01.783ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40532025-08-26T21:49:01.784ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 94.601003ms iteration = 7 status = {"errors":[],"finish_invoked_ok":[]}
40542025-08-26T21:49:01.786ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
40552025-08-26T21:49:01.786ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
40562025-08-26T21:49:01.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
40572025-08-26T21:49:01.787ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
40582025-08-26T21:49:01.787ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
40592025-08-26T21:49:01.787ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
40602025-08-26T21:49:01.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
40612025-08-26T21:49:01.788ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
40622025-08-26T21:49:01.788ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
40632025-08-26T21:49:01.788ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 0 version = 2
40642025-08-26T21:49:01.788ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
40652025-08-26T21:49:01.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
40662025-08-26T21:49:01.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
40672025-08-26T21:49:01.789ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
40682025-08-26T21:49:01.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
40692025-08-26T21:49:01.791ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-08-26T21:45:59.866217Z, time_modified: 2025-08-26T21:45:59.866217Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }) }
40702025-08-26T21:49:01.791ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 125.881098ms iteration = 4 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]}},"default_endpoint":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
40712025-08-26T21:49:01.792ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 116.984427ms iteration = 7 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
40722025-08-26T21:49:01.792ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 102.658114ms iteration = 7 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
40732025-08-26T21:49:01.792ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40742025-08-26T21:49:01.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40752025-08-26T21:49:01.793ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
40762025-08-26T21:49:01.793ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
40772025-08-26T21:49:01.793ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
40782025-08-26T21:49:01.793ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
40792025-08-26T21:49:01.793ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 118.731086ms iteration = 7 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
40802025-08-26T21:49:01.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
40812025-08-26T21:49:01.794ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40822025-08-26T21:49:01.795ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40832025-08-26T21:49:01.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40842025-08-26T21:49:01.797ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
40852025-08-26T21:49:01.797ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
40862025-08-26T21:49:01.797ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 109.189251ms iteration = 7 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
40872025-08-26T21:49:01.797ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40882025-08-26T21:49:01.802ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
40892025-08-26T21:49:01.802ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40902025-08-26T21:49:01.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40912025-08-26T21:49:01.804ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
40922025-08-26T21:49:01.805ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
40932025-08-26T21:49:01.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
40942025-08-26T21:49:01.809ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 119.718649ms iteration = 7 status = {"errors":[],"requests_created_ok":[]}
40952025-08-26T21:49:01.809ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 134.091113ms iteration = 7 status = 0
40962025-08-26T21:49:01.809ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 119.582516ms iteration = 7 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
40972025-08-26T21:49:01.810ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 4 version = 2
40982025-08-26T21:49:01.810ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
40992025-08-26T21:49:01.810ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-08-26 21:46:05.296205 UTC dns_group = external generation = 2
41002025-08-26T21:49:01.810ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 144.771693ms iteration = 4 status = {"generation":2}
41012025-08-26T21:49:01.827ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
41022025-08-26T21:49:01.827ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
41032025-08-26T21:49:01.828ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7940ccdd-ebbc-46ef-9e8f-491b8b5debd6", "content-length": "26", "date": "Tue, 26 Aug 2025 21:49:01 GMT"} })
41042025-08-26T21:49:01.828ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
41052025-08-26T21:49:01.837ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
41062025-08-26T21:49:01.886ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
41072025-08-26T21:49:01.886ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
41082025-08-26T21:49:01.887ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f0fe65d-9aaf-4258-9edb-cfc0d3c26dcd", "content-length": "372", "date": "Tue, 26 Aug 2025 21:49:01 GMT"} })
41092025-08-26T21:49:01.887ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
41102025-08-26T21:49:01.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41112025-08-26T21:49:01.891ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
41122025-08-26T21:49:01.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
41132025-08-26T21:49:01.897ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
41142025-08-26T21:49:01.897ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
41152025-08-26T21:49:01.898ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6dd5dbd-94bc-4e47-8a07-76a54c042013", "content-length": "26", "date": "Tue, 26 Aug 2025 21:49:01 GMT"} })
41162025-08-26T21:49:01.898ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
41172025-08-26T21:49:01.900ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
41182025-08-26T21:49:01.902ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
41192025-08-26T21:49:01.905ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
41202025-08-26T21:49:01.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
41212025-08-26T21:49:01.908ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41222025-08-26T21:49:01.911ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
41232025-08-26T21:49:01.911ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
41242025-08-26T21:49:01.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
41252025-08-26T21:49:01.913ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41262025-08-26T21:49:01.915ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
41272025-08-26T21:49:01.915ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
41282025-08-26T21:49:01.915ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "8e3959c3-c817-4c43-a1d1-28c246e1beaf", "date": "Tue, 26 Aug 2025 21:49:01 GMT"} })
41292025-08-26T21:49:01.915ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
41302025-08-26T21:49:01.915ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
41312025-08-26T21:49:01.915ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
41322025-08-26T21:49:01.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
41332025-08-26T21:49:01.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41342025-08-26T21:49:01.916ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
41352025-08-26T21:49:01.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
41362025-08-26T21:49:01.918ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
41372025-08-26T21:49:01.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41382025-08-26T21:49:01.920ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 245.679838ms iteration = 8 status = {"physical_disks_added":0}
41392025-08-26T21:49:01.921ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
41402025-08-26T21:49:01.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41412025-08-26T21:49:01.926ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
41422025-08-26T21:49:01.926ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
41432025-08-26T21:49:01.926ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "7221c14c-4d28-460d-8681-f0b3c7c168c8", "date": "Tue, 26 Aug 2025 21:49:01 GMT"} })
41442025-08-26T21:49:01.928ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 238.067196ms iteration = 7 status = {}
41452025-08-26T21:49:02.027ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
41462025-08-26T21:49:02.027ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
41472025-08-26T21:49:02.028ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "419aa90d-7a1b-498f-b604-f1d73b0ede26", "content-length": "26", "date": "Tue, 26 Aug 2025 21:49:01 GMT"} })
41482025-08-26T21:49:02.028ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
41492025-08-26T21:49:02.029ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
41502025-08-26T21:49:02.036ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
41512025-08-26T21:49:02.056ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
41522025-08-26T21:49:02.056ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 368.764242ms iteration = 4 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
41532025-08-26T21:49:02.058ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
41542025-08-26T21:49:02.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
41552025-08-26T21:49:02.060ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 395.295501ms iteration = 7 status = null
41562025-08-26T21:49:02.085ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 420.438387ms iteration = 7 status = {}
41572025-08-26T21:49:02.102ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
41582025-08-26T21:49:02.103ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f0dcb0f-4e1d-4849-95e6-db7701e22e77", "content-length": "691", "date": "Tue, 26 Aug 2025 21:49:01 GMT"} })
41592025-08-26T21:49:02.103ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
41602025-08-26T21:49:02.103ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
41612025-08-26T21:49:02.103ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
41622025-08-26T21:49:02.104ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 429.066892ms iteration = 7 status = {}
41632025-08-26T21:49:02.178ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
41642025-08-26T21:49:02.178ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "7607110f-49e6-4db3-a87d-771240293423", "date": "Tue, 26 Aug 2025 21:49:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
41652025-08-26T21:49:02.178ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
41662025-08-26T21:49:02.180ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "5fc8c415-b0d9-4f57-8fbb-032026bac622", "date": "Tue, 26 Aug 2025 21:49:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
41672025-08-26T21:49:02.180ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
41682025-08-26T21:49:02.182ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
41692025-08-26T21:49:02.182ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 517.335432ms iteration = 6 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
41702025-08-26T21:49:02.203ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
41712025-08-26T21:49:02.268ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
41722025-08-26T21:49:02.269ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "73f2e204-0e9d-4964-8a67-ffbe3e1b5f1a", "date": "Tue, 26 Aug 2025 21:49:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
41732025-08-26T21:49:02.269ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
41742025-08-26T21:49:02.270ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "9807b0e0-f1af-4731-8bed-49bfc29ac1d1", "date": "Tue, 26 Aug 2025 21:49:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
41752025-08-26T21:49:02.270ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
41762025-08-26T21:49:02.272ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "79d78d48-0468-476c-8763-ecc667eb5ff2", "date": "Tue, 26 Aug 2025 21:49:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
41772025-08-26T21:49:02.272ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
41782025-08-26T21:49:02.274ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
41792025-08-26T21:49:02.274ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 608.857765ms iteration = 6 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
41802025-08-26T21:49:02.275ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41812025-08-26T21:49:02.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41822025-08-26T21:49:02.278ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
41832025-08-26T21:49:02.278ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41842025-08-26T21:49:02.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41852025-08-26T21:49:02.292ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
41862025-08-26T21:49:02.292ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
41872025-08-26T21:49:02.292ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
41882025-08-26T21:49:02.292ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
41892025-08-26T21:49:02.292ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
41902025-08-26T21:49:02.292ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
41912025-08-26T21:49:02.292ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
41922025-08-26T21:49:02.292ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
41932025-08-26T21:49:02.292ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
41942025-08-26T21:49:02.292ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
41952025-08-26T21:49:02.293ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
41962025-08-26T21:49:02.293ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41972025-08-26T21:49:02.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41982025-08-26T21:49:02.296ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
41992025-08-26T21:49:02.296ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
42002025-08-26T21:49:02.416ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "5ad0bbca-ba3c-4e60-ba89-73de9a8aecf9", "date": "Tue, 26 Aug 2025 21:49:01 GMT"} })
42012025-08-26T21:49:02.416ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
42022025-08-26T21:49:02.416ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
42032025-08-26T21:49:02.416ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "9be250c0-9d12-4b5a-a69d-3fdeb2115b06", "content-length": "682", "date": "Tue, 26 Aug 2025 21:49:01 GMT"} })
42042025-08-26T21:49:02.416ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42052025-08-26T21:49:02.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42062025-08-26T21:49:02.419ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42072025-08-26T21:49:02.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42082025-08-26T21:49:02.422ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42092025-08-26T21:49:02.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42102025-08-26T21:49:02.425ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42112025-08-26T21:49:02.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42122025-08-26T21:49:02.432ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 757.23392ms iteration = 7 status = {}
42132025-08-26T21:49:05.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 403 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 97759a15-27bb-4bad-95ea-8f3b43013644 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
42142025-08-26T21:49:06.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 38 reason = Timeout
42152025-08-26T21:49:06.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
42162025-08-26T21:49:06.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
42172025-08-26T21:49:06.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42182025-08-26T21:49:06.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
42192025-08-26T21:49:06.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
42202025-08-26T21:49:06.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
42212025-08-26T21:49:06.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
42222025-08-26T21:49:06.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.037679ms iteration = 38 status = {"chicken_switches_updated":false}
42232025-08-26T21:49:11.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 39 reason = Timeout
42242025-08-26T21:49:11.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
42252025-08-26T21:49:11.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
42262025-08-26T21:49:11.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42272025-08-26T21:49:11.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
42282025-08-26T21:49:11.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
42292025-08-26T21:49:11.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
42302025-08-26T21:49:11.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
42312025-08-26T21:49:11.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.726141ms iteration = 39 status = {"chicken_switches_updated":false}
42322025-08-26T21:49:11.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 20 reason = Timeout
42332025-08-26T21:49:11.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42342025-08-26T21:49:11.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42352025-08-26T21:49:11.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42362025-08-26T21:49:11.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
42372025-08-26T21:49:11.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42382025-08-26T21:49:11.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42392025-08-26T21:49:11.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42402025-08-26T21:49:11.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42412025-08-26T21:49:11.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42422025-08-26T21:49:11.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
42432025-08-26T21:49:11.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42442025-08-26T21:49:11.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42452025-08-26T21:49:11.687ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 20 reason = Timeout
42462025-08-26T21:49:11.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
42472025-08-26T21:49:11.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
42482025-08-26T21:49:11.689ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
42492025-08-26T21:49:11.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
42502025-08-26T21:49:11.692ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.734937ms iteration = 20 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
42512025-08-26T21:49:11.705ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
42522025-08-26T21:49:11.706ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 30.514438ms iteration = 20 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
42532025-08-26T21:49:15.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 351 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 090c75d4-4442-48a3-a8eb-bed1c859b4eb response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
42542025-08-26T21:49:16.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 40 reason = Timeout
42552025-08-26T21:49:16.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
42562025-08-26T21:49:16.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
42572025-08-26T21:49:16.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42582025-08-26T21:49:16.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
42592025-08-26T21:49:16.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
42602025-08-26T21:49:16.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
42612025-08-26T21:49:16.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
42622025-08-26T21:49:16.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.746353ms iteration = 40 status = {"chicken_switches_updated":false}
42632025-08-26T21:49:21.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 41 reason = Timeout
42642025-08-26T21:49:21.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
42652025-08-26T21:49:21.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
42662025-08-26T21:49:21.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42672025-08-26T21:49:21.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
42682025-08-26T21:49:21.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
42692025-08-26T21:49:21.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
42702025-08-26T21:49:21.675ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
42712025-08-26T21:49:21.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 21 reason = Timeout
42722025-08-26T21:49:21.675ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.809416ms iteration = 41 status = {"chicken_switches_updated":false}
42732025-08-26T21:49:21.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42742025-08-26T21:49:21.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42752025-08-26T21:49:21.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42762025-08-26T21:49:21.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
42772025-08-26T21:49:21.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42782025-08-26T21:49:21.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42792025-08-26T21:49:21.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42802025-08-26T21:49:21.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42812025-08-26T21:49:21.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42822025-08-26T21:49:21.687ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
42832025-08-26T21:49:21.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42842025-08-26T21:49:21.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
42852025-08-26T21:49:21.687ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 21 reason = Timeout
42862025-08-26T21:49:21.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
42872025-08-26T21:49:21.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
42882025-08-26T21:49:21.689ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
42892025-08-26T21:49:21.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
42902025-08-26T21:49:21.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.407357ms iteration = 21 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
42912025-08-26T21:49:21.698ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
42922025-08-26T21:49:21.698ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.331076ms iteration = 21 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
42932025-08-26T21:49:25.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 612 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 22016d47-2d97-4651-ab0f-e44e83228bd9 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
42942025-08-26T21:49:26.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 42 reason = Timeout
42952025-08-26T21:49:26.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
42962025-08-26T21:49:26.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
42972025-08-26T21:49:26.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42982025-08-26T21:49:26.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
42992025-08-26T21:49:26.673ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
43002025-08-26T21:49:26.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
43012025-08-26T21:49:26.675ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
43022025-08-26T21:49:26.675ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.777066ms iteration = 42 status = {"chicken_switches_updated":false}
43032025-08-26T21:49:30.346ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46227 req_id = 129aaa66-9e2d-44bb-8099-f6c7e916032b roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
43042025-08-26T21:49:30.347ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46227 req_id = 129aaa66-9e2d-44bb-8099-f6c7e916032b resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
43052025-08-26T21:49:30.349ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3352 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46227 req_id = 129aaa66-9e2d-44bb-8099-f6c7e916032b response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
43062025-08-26T21:49:30.350ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46227 req_id = 29606fd5-0d9b-45af-b2c0-b8cf972e5b44 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
43072025-08-26T21:49:30.350ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46227 req_id = 29606fd5-0d9b-45af-b2c0-b8cf972e5b44 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
43082025-08-26T21:49:30.352ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2239 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:46227 req_id = 29606fd5-0d9b-45af-b2c0-b8cf972e5b44 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
43092025-08-26T21:49:31.665ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 8 reason = Timeout
43102025-08-26T21:49:31.665ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 8 reason = Timeout
43112025-08-26T21:49:31.665ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
43122025-08-26T21:49:31.665ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 8 reason = Timeout
43132025-08-26T21:49:31.665ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
43142025-08-26T21:49:31.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
43152025-08-26T21:49:31.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
43162025-08-26T21:49:31.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 43 reason = Timeout
43172025-08-26T21:49:31.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
43182025-08-26T21:49:31.668ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1075, tv_nsec: 525818386 } })
43192025-08-26T21:49:31.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
43202025-08-26T21:49:31.668ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.715551ms iteration = 8 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
43212025-08-26T21:49:31.668ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1075, tv_nsec: 526052871 } })
43222025-08-26T21:49:31.668ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
43232025-08-26T21:49:31.668ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
43242025-08-26T21:49:31.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43252025-08-26T21:49:31.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
43262025-08-26T21:49:31.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
43272025-08-26T21:49:31.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
43282025-08-26T21:49:31.675ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
43292025-08-26T21:49:31.675ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.10872ms iteration = 43 status = {"chicken_switches_updated":false}
43302025-08-26T21:49:31.675ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 8 reason = Timeout
43312025-08-26T21:49:31.675ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 9 reason = Timeout
43322025-08-26T21:49:31.675ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 8 reason = Timeout
43332025-08-26T21:49:31.675ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
43342025-08-26T21:49:31.675ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
43352025-08-26T21:49:31.675ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43362025-08-26T21:49:31.675ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 8 reason = Timeout
43372025-08-26T21:49:31.675ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
43382025-08-26T21:49:31.675ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 8 reason = Timeout
43392025-08-26T21:49:31.675ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
43402025-08-26T21:49:31.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 22 reason = Timeout
43412025-08-26T21:49:31.675ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 8 reason = Timeout
43422025-08-26T21:49:31.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43432025-08-26T21:49:31.675ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
43442025-08-26T21:49:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
43452025-08-26T21:49:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
43462025-08-26T21:49:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43472025-08-26T21:49:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
43482025-08-26T21:49:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
43492025-08-26T21:49:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43502025-08-26T21:49:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
43512025-08-26T21:49:31.676ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43522025-08-26T21:49:31.677ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43532025-08-26T21:49:31.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43542025-08-26T21:49:31.677ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43552025-08-26T21:49:31.677ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43562025-08-26T21:49:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
43572025-08-26T21:49:31.680ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
43582025-08-26T21:49:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
43592025-08-26T21:49:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
43602025-08-26T21:49:31.681ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43612025-08-26T21:49:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
43622025-08-26T21:49:31.681ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
43632025-08-26T21:49:31.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
43642025-08-26T21:49:31.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
43652025-08-26T21:49:31.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
43662025-08-26T21:49:31.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43672025-08-26T21:49:31.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
43682025-08-26T21:49:31.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
43692025-08-26T21:49:31.683ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
43702025-08-26T21:49:31.683ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
43712025-08-26T21:49:31.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
43722025-08-26T21:49:31.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
43732025-08-26T21:49:31.687ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 8 reason = Timeout
43742025-08-26T21:49:31.687ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
43752025-08-26T21:49:31.687ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 22 reason = Timeout
43762025-08-26T21:49:31.687ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 8 reason = Timeout
43772025-08-26T21:49:31.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
43782025-08-26T21:49:31.687ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
43792025-08-26T21:49:31.687ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
43802025-08-26T21:49:31.687ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1075, tv_nsec: 545202313 } })
43812025-08-26T21:49:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
43822025-08-26T21:49:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
43832025-08-26T21:49:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
43842025-08-26T21:49:31.688ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 8 reason = Timeout
43852025-08-26T21:49:31.688ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 8 reason = Timeout
43862025-08-26T21:49:31.688ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 13.79\u{b5}s iteration = 8 status = {"disabled":true,"errors":[],"sps":[]}
43872025-08-26T21:49:31.688ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 8 reason = Timeout
43882025-08-26T21:49:31.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
43892025-08-26T21:49:31.688ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 8 reason = Timeout
43902025-08-26T21:49:31.688ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
43912025-08-26T21:49:31.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
43922025-08-26T21:49:31.688ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 8 reason = Timeout
43932025-08-26T21:49:31.688ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
43942025-08-26T21:49:31.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
43952025-08-26T21:49:31.688ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
43962025-08-26T21:49:31.688ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 8 reason = Timeout
43972025-08-26T21:49:31.688ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
43982025-08-26T21:49:31.688ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 8 reason = Timeout
43992025-08-26T21:49:31.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
44002025-08-26T21:49:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
44012025-08-26T21:49:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
44022025-08-26T21:49:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
44032025-08-26T21:49:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
44042025-08-26T21:49:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
44052025-08-26T21:49:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
44062025-08-26T21:49:31.688ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44072025-08-26T21:49:31.689ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44082025-08-26T21:49:31.690ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44092025-08-26T21:49:31.690ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
44102025-08-26T21:49:31.690ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
44112025-08-26T21:49:31.690ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
44122025-08-26T21:49:31.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
44132025-08-26T21:49:31.691ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
44142025-08-26T21:49:31.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
44152025-08-26T21:49:31.691ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
44162025-08-26T21:49:31.691ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
44172025-08-26T21:49:31.691ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
44182025-08-26T21:49:31.692ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
44192025-08-26T21:49:31.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
44202025-08-26T21:49:31.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
44212025-08-26T21:49:31.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
44222025-08-26T21:49:31.692ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
44232025-08-26T21:49:31.693ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
44242025-08-26T21:49:31.693ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
44252025-08-26T21:49:31.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
44262025-08-26T21:49:31.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
44272025-08-26T21:49:31.693ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
44282025-08-26T21:49:31.693ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
44292025-08-26T21:49:31.694ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44302025-08-26T21:49:31.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
44312025-08-26T21:49:31.695ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 7.346586ms iteration = 8 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
44322025-08-26T21:49:31.695ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44332025-08-26T21:49:31.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44342025-08-26T21:49:31.696ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44352025-08-26T21:49:31.698ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
44362025-08-26T21:49:31.698ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
44372025-08-26T21:49:31.698ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44382025-08-26T21:49:31.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
44392025-08-26T21:49:31.699ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 10.842125ms iteration = 8 status = {"errors":[],"requests_created_ok":[]}
44402025-08-26T21:49:31.700ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
44412025-08-26T21:49:31.700ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 25.447554ms iteration = 8 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
44422025-08-26T21:49:31.701ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
44432025-08-26T21:49:31.701ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
44442025-08-26T21:49:31.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
44452025-08-26T21:49:31.704ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
44462025-08-26T21:49:31.704ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44472025-08-26T21:49:31.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
44482025-08-26T21:49:31.742ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
44492025-08-26T21:49:31.742ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 67.321368ms iteration = 22 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
44502025-08-26T21:49:31.754ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
44512025-08-26T21:49:31.754ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
44522025-08-26T21:49:31.754ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
44532025-08-26T21:49:31.754ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 66.171812ms iteration = 8 status = {"errors":[],"garbage_collect_requested":[]}
44542025-08-26T21:49:31.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
44552025-08-26T21:49:31.757ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
44562025-08-26T21:49:31.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
44572025-08-26T21:49:31.759ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 70.897808ms iteration = 8 status = {"errors":[],"finish_invoked_ok":[]}
44582025-08-26T21:49:31.759ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
44592025-08-26T21:49:31.759ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 72.27701ms iteration = 22 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
44602025-08-26T21:49:31.759ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
44612025-08-26T21:49:31.759ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
44622025-08-26T21:49:31.759ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 72.386372ms iteration = 8 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
44632025-08-26T21:49:31.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
44642025-08-26T21:49:31.761ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
44652025-08-26T21:49:31.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
44662025-08-26T21:49:31.763ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
44672025-08-26T21:49:31.763ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 75.951612ms iteration = 8 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
44682025-08-26T21:49:31.763ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 75.570274ms iteration = 8 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
44692025-08-26T21:49:31.767ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
44702025-08-26T21:49:31.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
44712025-08-26T21:49:31.769ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
44722025-08-26T21:49:31.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
44732025-08-26T21:49:31.771ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
44742025-08-26T21:49:31.771ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
44752025-08-26T21:49:31.771ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "878f4ef9-ec28-4587-986c-3efcf20f0b69", "content-length": "26", "date": "Tue, 26 Aug 2025 21:49:31 GMT"} })
44762025-08-26T21:49:31.771ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
44772025-08-26T21:49:31.772ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
44782025-08-26T21:49:31.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
44792025-08-26T21:49:31.773ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
44802025-08-26T21:49:31.776ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44812025-08-26T21:49:31.780ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
44822025-08-26T21:49:31.780ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
44832025-08-26T21:49:31.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
44842025-08-26T21:49:31.783ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 108.522997ms iteration = 8 status = 0
44852025-08-26T21:49:31.800ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
44862025-08-26T21:49:31.800ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
44872025-08-26T21:49:31.800ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff53713d-d2a4-447b-a8ea-10a915d14960", "content-length": "26", "date": "Tue, 26 Aug 2025 21:49:31 GMT"} })
44882025-08-26T21:49:31.800ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
44892025-08-26T21:49:31.803ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
44902025-08-26T21:49:31.813ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 138.38544ms iteration = 8 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
44912025-08-26T21:49:31.813ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
44922025-08-26T21:49:31.813ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
44932025-08-26T21:49:31.814ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "68542b1d-5153-4f9a-b997-1545dd93b1e8", "content-length": "26", "date": "Tue, 26 Aug 2025 21:49:31 GMT"} })
44942025-08-26T21:49:31.814ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
44952025-08-26T21:49:31.815ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
44962025-08-26T21:49:31.840ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
44972025-08-26T21:49:31.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
44982025-08-26T21:49:31.841ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
44992025-08-26T21:49:31.841ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
45002025-08-26T21:49:31.841ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e18a6118-d086-4fb9-8fb3-aa56070e4de2", "content-length": "372", "date": "Tue, 26 Aug 2025 21:49:31 GMT"} })
45012025-08-26T21:49:31.841ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
45022025-08-26T21:49:31.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
45032025-08-26T21:49:31.842ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45042025-08-26T21:49:31.845ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
45052025-08-26T21:49:31.845ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
45062025-08-26T21:49:31.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
45072025-08-26T21:49:31.847ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45082025-08-26T21:49:31.849ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
45092025-08-26T21:49:31.849ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
45102025-08-26T21:49:31.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
45112025-08-26T21:49:31.856ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 181.01043ms iteration = 9 status = {"physical_disks_added":0}
45122025-08-26T21:49:31.868ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
45132025-08-26T21:49:31.868ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
45142025-08-26T21:49:31.868ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "367876cf-519f-4924-b4c4-1a48f1aca330", "date": "Tue, 26 Aug 2025 21:49:31 GMT"} })
45152025-08-26T21:49:31.868ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
45162025-08-26T21:49:31.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
45172025-08-26T21:49:31.872ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
45182025-08-26T21:49:31.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
45192025-08-26T21:49:31.875ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
45202025-08-26T21:49:31.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
45212025-08-26T21:49:31.879ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
45222025-08-26T21:49:31.879ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
45232025-08-26T21:49:31.880ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "f4c29860-6764-4786-a164-f98b5f557a81", "date": "Tue, 26 Aug 2025 21:49:31 GMT"} })
45242025-08-26T21:49:31.881ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 193.357389ms iteration = 8 status = {}
45252025-08-26T21:49:31.892ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 226.69827ms iteration = 8 status = {}
45262025-08-26T21:49:31.925ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
45272025-08-26T21:49:31.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
45282025-08-26T21:49:31.927ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 261.611288ms iteration = 8 status = null
45292025-08-26T21:49:32.006ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
45302025-08-26T21:49:32.011ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b5c96a4-50e8-423a-b73b-d0eed6983d90", "content-length": "691", "date": "Tue, 26 Aug 2025 21:49:31 GMT"} })
45312025-08-26T21:49:32.011ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
45322025-08-26T21:49:32.011ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
45332025-08-26T21:49:32.011ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
45342025-08-26T21:49:32.013ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 338.284355ms iteration = 8 status = {}
45352025-08-26T21:49:32.060ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45362025-08-26T21:49:32.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45372025-08-26T21:49:32.063ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
45382025-08-26T21:49:32.063ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45392025-08-26T21:49:32.063ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45402025-08-26T21:49:32.076ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
45412025-08-26T21:49:32.076ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
45422025-08-26T21:49:32.076ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
45432025-08-26T21:49:32.076ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
45442025-08-26T21:49:32.076ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
45452025-08-26T21:49:32.076ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
45462025-08-26T21:49:32.076ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
45472025-08-26T21:49:32.076ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
45482025-08-26T21:49:32.077ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
45492025-08-26T21:49:32.077ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
45502025-08-26T21:49:32.077ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
45512025-08-26T21:49:32.077ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45522025-08-26T21:49:32.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45532025-08-26T21:49:32.079ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
45542025-08-26T21:49:32.079ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
45552025-08-26T21:49:32.210ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "864f0d21-6e75-40da-9157-a81d01f0d337", "date": "Tue, 26 Aug 2025 21:49:31 GMT"} })
45562025-08-26T21:49:32.210ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
45572025-08-26T21:49:32.210ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
45582025-08-26T21:49:32.210ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "5191510e-bbc0-47de-bf65-1f9e899e9bf5", "content-length": "682", "date": "Tue, 26 Aug 2025 21:49:31 GMT"} })
45592025-08-26T21:49:32.210ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45602025-08-26T21:49:32.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45612025-08-26T21:49:32.214ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45622025-08-26T21:49:32.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45632025-08-26T21:49:32.218ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45642025-08-26T21:49:32.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45652025-08-26T21:49:32.221ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45662025-08-26T21:49:32.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45672025-08-26T21:49:32.229ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 554.121951ms iteration = 8 status = {}
45682025-08-26T21:49:32.261ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 1
45692025-08-26T21:49:32.261ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221
45702025-08-26T21:49:32.294ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
45712025-08-26T21:49:32.294ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::a]:50689
45722025-08-26T21:49:32.294ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:50689 req_id = 616b0022-6cee-4bc6-9431-bc8f4c9905d1 roles = RoleSet { roles: {} } uri = /metrics/producers
45732025-08-26T21:49:32.295ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:50689 req_id = 616b0022-6cee-4bc6-9431-bc8f4c9905d1 resource = Database result = Ok(()) uri = /metrics/producers
45742025-08-26T21:49:32.328ZINFOnexus: assigned collector to new producer collector_id = f843ff0e-eef8-49c4-b428-23e15df1def3 file = nexus/src/app/oximeter.rs:76 producer_id = 4996dcc4-c530-4f64-b407-fa372462bd9e
45752025-08-26T21:49:32.328ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 33897 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:50689 req_id = 616b0022-6cee-4bc6-9431-bc8f4c9905d1 response_code = 201 uri = /metrics/producers
45762025-08-26T21:49:32.328ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "616b0022-6cee-4bc6-9431-bc8f4c9905d1", "content-length": "41", "date": "Tue, 26 Aug 2025 21:49:32 GMT"} })
45772025-08-26T21:49:32.329ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
45782025-08-26T21:49:35.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 572 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 3b4dd353-1418-4d62-81a2-9f2e4fe6c89d response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
45792025-08-26T21:49:36.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 44 reason = Timeout
45802025-08-26T21:49:36.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
45812025-08-26T21:49:36.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
45822025-08-26T21:49:36.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45832025-08-26T21:49:36.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
45842025-08-26T21:49:36.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
45852025-08-26T21:49:36.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
45862025-08-26T21:49:36.675ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
45872025-08-26T21:49:36.675ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.611492ms iteration = 44 status = {"chicken_switches_updated":false}
45882025-08-26T21:49:41.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 45 reason = Timeout
45892025-08-26T21:49:41.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
45902025-08-26T21:49:41.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
45912025-08-26T21:49:41.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45922025-08-26T21:49:41.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
45932025-08-26T21:49:41.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
45942025-08-26T21:49:41.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
45952025-08-26T21:49:41.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
45962025-08-26T21:49:41.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.617239ms iteration = 45 status = {"chicken_switches_updated":false}
45972025-08-26T21:49:41.676ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 23 reason = Timeout
45982025-08-26T21:49:41.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45992025-08-26T21:49:41.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46002025-08-26T21:49:41.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46012025-08-26T21:49:41.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
46022025-08-26T21:49:41.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46032025-08-26T21:49:41.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46042025-08-26T21:49:41.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46052025-08-26T21:49:41.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46062025-08-26T21:49:41.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46072025-08-26T21:49:41.687ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 23 reason = Timeout
46082025-08-26T21:49:41.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
46092025-08-26T21:49:41.688ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
46102025-08-26T21:49:41.688ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46112025-08-26T21:49:41.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
46122025-08-26T21:49:41.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46132025-08-26T21:49:41.689ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
46142025-08-26T21:49:41.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
46152025-08-26T21:49:41.691ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.796566ms iteration = 23 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
46162025-08-26T21:49:41.703ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
46172025-08-26T21:49:41.703ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.707014ms iteration = 23 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
46182025-08-26T21:49:45.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 357 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 98213cd6-abe5-4da2-afd8-e7fc3da84e80 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
46192025-08-26T21:49:46.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 46 reason = Timeout
46202025-08-26T21:49:46.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
46212025-08-26T21:49:46.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
46222025-08-26T21:49:46.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46232025-08-26T21:49:46.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
46242025-08-26T21:49:46.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
46252025-08-26T21:49:46.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
46262025-08-26T21:49:46.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
46272025-08-26T21:49:46.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.594051ms iteration = 46 status = {"chicken_switches_updated":false}
46282025-08-26T21:49:51.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 47 reason = Timeout
46292025-08-26T21:49:51.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
46302025-08-26T21:49:51.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
46312025-08-26T21:49:51.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46322025-08-26T21:49:51.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
46332025-08-26T21:49:51.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
46342025-08-26T21:49:51.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
46352025-08-26T21:49:51.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
46362025-08-26T21:49:51.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.727871ms iteration = 47 status = {"chicken_switches_updated":false}
46372025-08-26T21:49:51.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 24 reason = Timeout
46382025-08-26T21:49:51.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46392025-08-26T21:49:51.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46402025-08-26T21:49:51.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46412025-08-26T21:49:51.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
46422025-08-26T21:49:51.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46432025-08-26T21:49:51.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46442025-08-26T21:49:51.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46452025-08-26T21:49:51.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46462025-08-26T21:49:51.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46472025-08-26T21:49:51.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
46482025-08-26T21:49:51.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46492025-08-26T21:49:51.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
46502025-08-26T21:49:51.687ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 24 reason = Timeout
46512025-08-26T21:49:51.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
46522025-08-26T21:49:51.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
46532025-08-26T21:49:51.699ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
46542025-08-26T21:49:51.699ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.188566ms iteration = 24 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
46552025-08-26T21:49:51.728ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
46562025-08-26T21:49:51.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
46572025-08-26T21:49:51.731ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 43.584722ms iteration = 24 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
46582025-08-26T21:49:55.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 604 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 19f6440d-ac43-45a8-a805-77a6db3ca804 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
46592025-08-26T21:49:56.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 48 reason = Timeout
46602025-08-26T21:49:56.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
46612025-08-26T21:49:56.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
46622025-08-26T21:49:56.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46632025-08-26T21:49:56.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
46642025-08-26T21:49:56.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
46652025-08-26T21:49:56.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
46662025-08-26T21:49:56.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
46672025-08-26T21:49:56.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.042519ms iteration = 48 status = {"chicken_switches_updated":false}
46682025-08-26T21:50:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 5 reason = Timeout
46692025-08-26T21:50:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 5 reason = Timeout
46702025-08-26T21:50:01.665ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
46712025-08-26T21:50:01.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
46722025-08-26T21:50:01.665ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 9 reason = Timeout
46732025-08-26T21:50:01.665ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 9 reason = Timeout
46742025-08-26T21:50:01.665ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
46752025-08-26T21:50:01.665ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 9 reason = Timeout
46762025-08-26T21:50:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 7 reason = Timeout
46772025-08-26T21:50:01.665ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
46782025-08-26T21:50:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 7 reason = Timeout
46792025-08-26T21:50:01.665ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 5 reason = Timeout
46802025-08-26T21:50:01.665ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:40:01.665636763 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
46812025-08-26T21:50:01.665ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
46822025-08-26T21:50:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 5 reason = Timeout
46832025-08-26T21:50:01.665ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
46842025-08-26T21:50:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 5 reason = Timeout
46852025-08-26T21:50:01.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
46862025-08-26T21:50:01.665ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 5 reason = Timeout
46872025-08-26T21:50:01.665ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
46882025-08-26T21:50:01.666ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 6 reason = Timeout
46892025-08-26T21:50:01.666ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
46902025-08-26T21:50:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
46912025-08-26T21:50:01.666ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 18.271\u{b5}s iteration = 6 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
46922025-08-26T21:50:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
46932025-08-26T21:50:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
46942025-08-26T21:50:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
46952025-08-26T21:50:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
46962025-08-26T21:50:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
46972025-08-26T21:50:01.666ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1105, tv_nsec: 523836762 } })
46982025-08-26T21:50:01.666ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1105, tv_nsec: 523851662 } })
46992025-08-26T21:50:01.666ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
47002025-08-26T21:50:01.666ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.143436ms iteration = 5 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
47012025-08-26T21:50:01.666ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
47022025-08-26T21:50:01.666ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
47032025-08-26T21:50:01.666ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.166996ms iteration = 5 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
47042025-08-26T21:50:01.666ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
47052025-08-26T21:50:01.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
47062025-08-26T21:50:01.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
47072025-08-26T21:50:01.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 49 reason = Timeout
47082025-08-26T21:50:01.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
47092025-08-26T21:50:01.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
47102025-08-26T21:50:01.667ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47112025-08-26T21:50:01.668ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1105, tv_nsec: 525876787 } })
47122025-08-26T21:50:01.668ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1105, tv_nsec: 525887108 } })
47132025-08-26T21:50:01.668ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.727492ms iteration = 9 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
47142025-08-26T21:50:01.668ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:40:01.665636763 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
47152025-08-26T21:50:01.668ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
47162025-08-26T21:50:01.668ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 2.921996ms iteration = 5 status = {"expiration":"2025-08-26T21:40:01.665636763Z","pruned":[]}
47172025-08-26T21:50:01.668ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
47182025-08-26T21:50:01.668ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47192025-08-26T21:50:01.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47202025-08-26T21:50:01.668ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47212025-08-26T21:50:01.670ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
47222025-08-26T21:50:01.670ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
47232025-08-26T21:50:01.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
47242025-08-26T21:50:01.672ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
47252025-08-26T21:50:01.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
47262025-08-26T21:50:01.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
47272025-08-26T21:50:01.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
47282025-08-26T21:50:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
47292025-08-26T21:50:01.674ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
47302025-08-26T21:50:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
47312025-08-26T21:50:01.674ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47322025-08-26T21:50:01.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 7 reason = Timeout
47332025-08-26T21:50:01.674ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
47342025-08-26T21:50:01.674ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 23.94\u{b5}s iteration = 7 status = "Disabled"
47352025-08-26T21:50:01.675ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 5 reason = Timeout
47362025-08-26T21:50:01.675ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
47372025-08-26T21:50:01.675ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
47382025-08-26T21:50:01.675ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 10 reason = Timeout
47392025-08-26T21:50:01.675ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 9 reason = Timeout
47402025-08-26T21:50:01.675ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
47412025-08-26T21:50:01.675ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
47422025-08-26T21:50:01.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 25 reason = Timeout
47432025-08-26T21:50:01.675ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 9 reason = Timeout
47442025-08-26T21:50:01.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47452025-08-26T21:50:01.675ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
47462025-08-26T21:50:01.675ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 9 reason = Timeout
47472025-08-26T21:50:01.675ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
47482025-08-26T21:50:01.675ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 9 reason = Timeout
47492025-08-26T21:50:01.675ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
47502025-08-26T21:50:01.675ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 9 reason = Timeout
47512025-08-26T21:50:01.675ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
47522025-08-26T21:50:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
47532025-08-26T21:50:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
47542025-08-26T21:50:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
47552025-08-26T21:50:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
47562025-08-26T21:50:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
47572025-08-26T21:50:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
47582025-08-26T21:50:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
47592025-08-26T21:50:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
47602025-08-26T21:50:01.676ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
47612025-08-26T21:50:01.676ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 9.444873ms iteration = 49 status = {"chicken_switches_updated":false}
47622025-08-26T21:50:01.677ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47632025-08-26T21:50:01.678ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47642025-08-26T21:50:01.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47652025-08-26T21:50:01.678ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47662025-08-26T21:50:01.678ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47672025-08-26T21:50:01.678ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47682025-08-26T21:50:01.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
47692025-08-26T21:50:01.680ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
47702025-08-26T21:50:01.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
47712025-08-26T21:50:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
47722025-08-26T21:50:01.683ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
47732025-08-26T21:50:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
47742025-08-26T21:50:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
47752025-08-26T21:50:01.683ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
47762025-08-26T21:50:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
47772025-08-26T21:50:01.683ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
47782025-08-26T21:50:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
47792025-08-26T21:50:01.683ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
47802025-08-26T21:50:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
47812025-08-26T21:50:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
47822025-08-26T21:50:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
47832025-08-26T21:50:01.684ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
47842025-08-26T21:50:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
47852025-08-26T21:50:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
47862025-08-26T21:50:01.684ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
47872025-08-26T21:50:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
47882025-08-26T21:50:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
47892025-08-26T21:50:01.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47902025-08-26T21:50:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
47912025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 5 reason = Timeout
47922025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
47932025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
47942025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 9 reason = Timeout
47952025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
47962025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
47972025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
47982025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 9 reason = Timeout
47992025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
48002025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
48012025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 25 reason = Timeout
48022025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
48032025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
48042025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 5 reason = Timeout
48052025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
48062025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
48072025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 5 reason = Timeout
48082025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
48092025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 9 reason = Timeout
48102025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
48112025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 9 reason = Timeout
48122025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
48132025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 9 reason = Timeout
48142025-08-26T21:50:01.691ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
48152025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48162025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
48172025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 5 reason = Timeout
48182025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
48192025-08-26T21:50:01.691ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 9 reason = Timeout
48202025-08-26T21:50:01.692ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 5 reason = Timeout
48212025-08-26T21:50:01.692ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 7.5\u{b5}s iteration = 9 status = {"disabled":true,"errors":[],"sps":[]}
48222025-08-26T21:50:01.692ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 9 reason = Timeout
48232025-08-26T21:50:01.692ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
48242025-08-26T21:50:01.692ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
48252025-08-26T21:50:01.692ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 9 reason = Timeout
48262025-08-26T21:50:01.692ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
48272025-08-26T21:50:01.692ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 9 reason = Timeout
48282025-08-26T21:50:01.692ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
48292025-08-26T21:50:01.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
48302025-08-26T21:50:01.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
48312025-08-26T21:50:01.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48322025-08-26T21:50:01.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
48332025-08-26T21:50:01.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
48342025-08-26T21:50:01.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
48352025-08-26T21:50:01.692ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1105, tv_nsec: 547101036 } })
48362025-08-26T21:50:01.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
48372025-08-26T21:50:01.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
48382025-08-26T21:50:01.692ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
48392025-08-26T21:50:01.692ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
48402025-08-26T21:50:01.692ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 3.436738ms iteration = 5 status = {"errors":[],"found_port_ok":[]}
48412025-08-26T21:50:01.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
48422025-08-26T21:50:01.692ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 2.947096ms iteration = 9 status = {"errors":[],"garbage_collect_requested":[]}
48432025-08-26T21:50:01.692ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48442025-08-26T21:50:01.692ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48452025-08-26T21:50:01.694ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
48462025-08-26T21:50:01.694ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 5.417743ms iteration = 5 status = {"by_rx":{},"error":null}
48472025-08-26T21:50:01.694ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
48482025-08-26T21:50:01.695ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 6.286131ms iteration = 9 status = {"errors":[],"finish_invoked_ok":[]}
48492025-08-26T21:50:01.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
48502025-08-26T21:50:01.696ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
48512025-08-26T21:50:01.696ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
48522025-08-26T21:50:01.696ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 7.228663ms iteration = 5 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
48532025-08-26T21:50:01.696ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
48542025-08-26T21:50:01.696ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
48552025-08-26T21:50:01.696ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
48562025-08-26T21:50:01.696ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
48572025-08-26T21:50:01.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
48582025-08-26T21:50:01.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
48592025-08-26T21:50:01.698ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
48602025-08-26T21:50:01.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
48612025-08-26T21:50:01.699ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
48622025-08-26T21:50:01.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
48632025-08-26T21:50:01.700ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
48642025-08-26T21:50:01.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
48652025-08-26T21:50:01.702ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 12.224875ms iteration = 9 status = {"errors":[],"requests_created_ok":[]}
48662025-08-26T21:50:01.703ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
48672025-08-26T21:50:01.703ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
48682025-08-26T21:50:01.703ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 27.94384ms iteration = 5 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
48692025-08-26T21:50:01.703ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
48702025-08-26T21:50:01.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
48712025-08-26T21:50:01.704ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48722025-08-26T21:50:01.705ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 16.82389ms iteration = 5 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
48732025-08-26T21:50:01.707ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
48742025-08-26T21:50:01.707ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
48752025-08-26T21:50:01.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
48762025-08-26T21:50:01.717ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
48772025-08-26T21:50:01.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
48782025-08-26T21:50:01.730ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
48792025-08-26T21:50:01.731ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 43.177953ms iteration = 9 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
48802025-08-26T21:50:01.734ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
48812025-08-26T21:50:01.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
48822025-08-26T21:50:01.736ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48832025-08-26T21:50:01.740ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
48842025-08-26T21:50:01.740ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
48852025-08-26T21:50:01.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
48862025-08-26T21:50:01.744ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48872025-08-26T21:50:01.746ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
48882025-08-26T21:50:01.747ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
48892025-08-26T21:50:01.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
48902025-08-26T21:50:01.764ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
48912025-08-26T21:50:01.764ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
48922025-08-26T21:50:01.764ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
48932025-08-26T21:50:01.764ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
48942025-08-26T21:50:01.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
48952025-08-26T21:50:01.778ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48962025-08-26T21:50:01.783ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48972025-08-26T21:50:01.783ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
48982025-08-26T21:50:01.783ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
48992025-08-26T21:50:01.783ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
49002025-08-26T21:50:01.783ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
49012025-08-26T21:50:01.783ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
49022025-08-26T21:50:01.783ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
49032025-08-26T21:50:01.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
49042025-08-26T21:50:01.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
49052025-08-26T21:50:01.787ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 111.694168ms iteration = 10 status = {"physical_disks_added":0}
49062025-08-26T21:50:01.787ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 98.837628ms iteration = 9 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
49072025-08-26T21:50:01.788ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
49082025-08-26T21:50:01.788ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
49092025-08-26T21:50:01.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
49102025-08-26T21:50:01.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
49112025-08-26T21:50:01.788ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49122025-08-26T21:50:01.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
49132025-08-26T21:50:01.792ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
49142025-08-26T21:50:01.792ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49152025-08-26T21:50:01.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49162025-08-26T21:50:01.792ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49172025-08-26T21:50:01.793ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49182025-08-26T21:50:01.793ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
49192025-08-26T21:50:01.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
49202025-08-26T21:50:01.796ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
49212025-08-26T21:50:01.796ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
49222025-08-26T21:50:01.796ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
49232025-08-26T21:50:01.796ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 107.943683ms iteration = 9 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
49242025-08-26T21:50:01.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
49252025-08-26T21:50:01.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
49262025-08-26T21:50:01.797ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49272025-08-26T21:50:01.797ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
49282025-08-26T21:50:01.797ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
49292025-08-26T21:50:01.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
49302025-08-26T21:50:01.799ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
49312025-08-26T21:50:01.799ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
49322025-08-26T21:50:01.799ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 111.34402ms iteration = 25 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
49332025-08-26T21:50:01.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
49342025-08-26T21:50:01.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
49352025-08-26T21:50:01.799ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
49362025-08-26T21:50:01.799ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 0 version = 2
49372025-08-26T21:50:01.799ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
49382025-08-26T21:50:01.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
49392025-08-26T21:50:01.800ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
49402025-08-26T21:50:01.800ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 124.524668ms iteration = 9 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
49412025-08-26T21:50:01.800ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
49422025-08-26T21:50:01.800ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49432025-08-26T21:50:01.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49442025-08-26T21:50:01.801ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
49452025-08-26T21:50:01.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
49462025-08-26T21:50:01.801ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49472025-08-26T21:50:01.802ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 126.185835ms iteration = 9 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
49482025-08-26T21:50:01.803ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49492025-08-26T21:50:01.805ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
49502025-08-26T21:50:01.805ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
49512025-08-26T21:50:01.805ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
49522025-08-26T21:50:01.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
49532025-08-26T21:50:01.807ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
49542025-08-26T21:50:01.807ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
49552025-08-26T21:50:01.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
49562025-08-26T21:50:01.808ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
49572025-08-26T21:50:01.808ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
49582025-08-26T21:50:01.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
49592025-08-26T21:50:01.808ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
49602025-08-26T21:50:01.808ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
49612025-08-26T21:50:01.808ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
49622025-08-26T21:50:01.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
49632025-08-26T21:50:01.809ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
49642025-08-26T21:50:01.809ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 134.121833ms iteration = 25 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
49652025-08-26T21:50:01.810ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 4 version = 2
49662025-08-26T21:50:01.810ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
49672025-08-26T21:50:01.810ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-08-26 21:46:05.296205 UTC dns_group = external generation = 2
49682025-08-26T21:50:01.810ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 144.768273ms iteration = 5 status = {"generation":2}
49692025-08-26T21:50:01.810ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 134.760678ms iteration = 9 status = 0
49702025-08-26T21:50:01.811ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
49712025-08-26T21:50:01.811ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
49722025-08-26T21:50:01.811ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 124.227051ms iteration = 9 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
49732025-08-26T21:50:01.823ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-08-26T21:45:59.866217Z, time_modified: 2025-08-26T21:45:59.866217Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }) }
49742025-08-26T21:50:01.823ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 158.077823ms iteration = 5 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]}},"default_endpoint":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
49752025-08-26T21:50:01.841ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
49762025-08-26T21:50:01.841ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
49772025-08-26T21:50:01.841ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
49782025-08-26T21:50:01.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
49792025-08-26T21:50:01.842ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
49802025-08-26T21:50:01.844ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49812025-08-26T21:50:01.848ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
49822025-08-26T21:50:01.848ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
49832025-08-26T21:50:01.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
49842025-08-26T21:50:01.880ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
49852025-08-26T21:50:01.880ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
49862025-08-26T21:50:01.880ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-08-26 21:46:04.065217 UTC dns_group = internal generation = 1
49872025-08-26T21:50:01.880ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 215.306163ms iteration = 5 status = {"generation":1}
49882025-08-26T21:50:01.894ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
49892025-08-26T21:50:01.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
49902025-08-26T21:50:01.929ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
49912025-08-26T21:50:01.929ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
49922025-08-26T21:50:01.929ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "53f252e4-ab17-4bb5-bba9-b8bbf9dbfff3", "content-length": "26", "date": "Tue, 26 Aug 2025 21:50:01 GMT"} })
49932025-08-26T21:50:01.929ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
49942025-08-26T21:50:01.931ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
49952025-08-26T21:50:01.962ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
49962025-08-26T21:50:01.983ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
49972025-08-26T21:50:01.983ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
49982025-08-26T21:50:01.983ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "26561693-2171-4603-8938-8bb57d4c38c0", "content-length": "26", "date": "Tue, 26 Aug 2025 21:50:01 GMT"} })
49992025-08-26T21:50:01.983ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
50002025-08-26T21:50:01.984ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
50012025-08-26T21:50:01.984ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "dbe5f08e-a42f-4353-ab50-c05185000afe", "content-length": "691", "date": "Tue, 26 Aug 2025 21:50:01 GMT"} })
50022025-08-26T21:50:01.984ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
50032025-08-26T21:50:01.984ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
50042025-08-26T21:50:01.984ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
50052025-08-26T21:50:01.985ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
50062025-08-26T21:50:01.986ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 311.036601ms iteration = 9 status = {}
50072025-08-26T21:50:02.025ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
50082025-08-26T21:50:02.025ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
50092025-08-26T21:50:02.025ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e6b32c1-06b6-4c86-8302-d8f332113beb", "content-length": "26", "date": "Tue, 26 Aug 2025 21:50:01 GMT"} })
50102025-08-26T21:50:02.025ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
50112025-08-26T21:50:02.027ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
50122025-08-26T21:50:02.038ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
50132025-08-26T21:50:02.038ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 351.838801ms iteration = 5 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
50142025-08-26T21:50:02.043ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
50152025-08-26T21:50:02.133ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
50162025-08-26T21:50:02.133ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
50172025-08-26T21:50:02.134ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "07aa029b-a97d-44fc-a7e7-f35dbd22075b", "content-length": "372", "date": "Tue, 26 Aug 2025 21:50:01 GMT"} })
50182025-08-26T21:50:02.134ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
50192025-08-26T21:50:02.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
50202025-08-26T21:50:02.148ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
50212025-08-26T21:50:02.148ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
50222025-08-26T21:50:02.148ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "c160abff-cf98-4bb5-bb3f-b04a482afcfe", "date": "Tue, 26 Aug 2025 21:50:01 GMT"} })
50232025-08-26T21:50:02.148ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
50242025-08-26T21:50:02.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
50252025-08-26T21:50:02.151ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
50262025-08-26T21:50:02.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
50272025-08-26T21:50:02.154ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
50282025-08-26T21:50:02.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
50292025-08-26T21:50:02.156ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
50302025-08-26T21:50:02.156ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
50312025-08-26T21:50:02.156ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "84c5e559-a4fa-4223-8a05-0e1a20063c86", "date": "Tue, 26 Aug 2025 21:50:02 GMT"} })
50322025-08-26T21:50:02.158ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 469.605626ms iteration = 9 status = {}
50332025-08-26T21:50:02.160ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 494.662911ms iteration = 9 status = {}
50342025-08-26T21:50:02.180ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
50352025-08-26T21:50:02.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
50362025-08-26T21:50:02.181ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
50372025-08-26T21:50:02.181ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "8a7d49a0-1815-463e-84a7-148edc7f37f5", "date": "Tue, 26 Aug 2025 21:50:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
50382025-08-26T21:50:02.181ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
50392025-08-26T21:50:02.182ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 516.427462ms iteration = 9 status = null
50402025-08-26T21:50:02.182ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
50412025-08-26T21:50:02.182ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "d715d2e9-9d2f-48b8-beff-17be66c17dd9", "date": "Tue, 26 Aug 2025 21:50:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
50422025-08-26T21:50:02.182ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
50432025-08-26T21:50:02.184ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "259e6736-b8df-469a-87b8-c5d1b5e886a2", "date": "Tue, 26 Aug 2025 21:50:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
50442025-08-26T21:50:02.184ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
50452025-08-26T21:50:02.185ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "18ea0565-1d5d-4dfe-a372-b497fc94328c", "date": "Tue, 26 Aug 2025 21:50:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
50462025-08-26T21:50:02.185ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
50472025-08-26T21:50:02.186ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "d5fbcd52-56e7-4ff0-8c2c-0d1c99b5d74b", "date": "Tue, 26 Aug 2025 21:50:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
50482025-08-26T21:50:02.186ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
50492025-08-26T21:50:02.188ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
50502025-08-26T21:50:02.188ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 522.954668ms iteration = 7 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
50512025-08-26T21:50:02.189ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
50522025-08-26T21:50:02.189ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 523.749917ms iteration = 7 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
50532025-08-26T21:50:02.247ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50542025-08-26T21:50:02.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50552025-08-26T21:50:02.250ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
50562025-08-26T21:50:02.250ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50572025-08-26T21:50:02.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50582025-08-26T21:50:02.260ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
50592025-08-26T21:50:02.260ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
50602025-08-26T21:50:02.261ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
50612025-08-26T21:50:02.261ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
50622025-08-26T21:50:02.261ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
50632025-08-26T21:50:02.261ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
50642025-08-26T21:50:02.261ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
50652025-08-26T21:50:02.261ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
50662025-08-26T21:50:02.261ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
50672025-08-26T21:50:02.261ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
50682025-08-26T21:50:02.261ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
50692025-08-26T21:50:02.261ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50702025-08-26T21:50:02.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50712025-08-26T21:50:02.263ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
50722025-08-26T21:50:02.263ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
50732025-08-26T21:50:02.459ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "6feac696-6e11-4cd5-bf5a-2dd42cb34528", "date": "Tue, 26 Aug 2025 21:50:02 GMT"} })
50742025-08-26T21:50:02.459ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
50752025-08-26T21:50:02.459ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
50762025-08-26T21:50:02.460ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "96052e7e-5189-4be2-9c78-69bea8dc5338", "content-length": "682", "date": "Tue, 26 Aug 2025 21:50:01 GMT"} })
50772025-08-26T21:50:02.460ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50782025-08-26T21:50:02.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50792025-08-26T21:50:02.462ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50802025-08-26T21:50:02.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50812025-08-26T21:50:02.466ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50822025-08-26T21:50:02.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50832025-08-26T21:50:02.468ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50842025-08-26T21:50:02.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50852025-08-26T21:50:02.474ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 799.101944ms iteration = 9 status = {}
50862025-08-26T21:50:05.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 389 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 422aa9d6-72dc-4563-9785-3963ca6f66b9 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
50872025-08-26T21:50:06.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 50 reason = Timeout
50882025-08-26T21:50:06.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
50892025-08-26T21:50:06.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
50902025-08-26T21:50:06.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50912025-08-26T21:50:06.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
50922025-08-26T21:50:06.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
50932025-08-26T21:50:06.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
50942025-08-26T21:50:06.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
50952025-08-26T21:50:06.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.318855ms iteration = 50 status = {"chicken_switches_updated":false}
50962025-08-26T21:50:11.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 51 reason = Timeout
50972025-08-26T21:50:11.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
50982025-08-26T21:50:11.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
50992025-08-26T21:50:11.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51002025-08-26T21:50:11.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
51012025-08-26T21:50:11.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
51022025-08-26T21:50:11.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
51032025-08-26T21:50:11.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
51042025-08-26T21:50:11.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.856874ms iteration = 51 status = {"chicken_switches_updated":false}
51052025-08-26T21:50:11.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 26 reason = Timeout
51062025-08-26T21:50:11.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51072025-08-26T21:50:11.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51082025-08-26T21:50:11.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51092025-08-26T21:50:11.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
51102025-08-26T21:50:11.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51112025-08-26T21:50:11.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51122025-08-26T21:50:11.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51132025-08-26T21:50:11.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51142025-08-26T21:50:11.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51152025-08-26T21:50:11.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
51162025-08-26T21:50:11.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51172025-08-26T21:50:11.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51182025-08-26T21:50:11.687ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 26 reason = Timeout
51192025-08-26T21:50:11.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51202025-08-26T21:50:11.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51212025-08-26T21:50:11.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51222025-08-26T21:50:11.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51232025-08-26T21:50:11.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.211492ms iteration = 26 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
51242025-08-26T21:50:11.697ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
51252025-08-26T21:50:11.697ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.62098ms iteration = 26 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
51262025-08-26T21:50:15.345ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:59936
51272025-08-26T21:50:15.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = fda3a390-ff3c-49d2-9446-97534fd531f6 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
51282025-08-26T21:50:15.346ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = fda3a390-ff3c-49d2-9446-97534fd531f6 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
51292025-08-26T21:50:15.348ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3176 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = fda3a390-ff3c-49d2-9446-97534fd531f6 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
51302025-08-26T21:50:15.349ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 11105f09-320a-477b-931b-6d15855c6c7e roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
51312025-08-26T21:50:15.349ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 11105f09-320a-477b-931b-6d15855c6c7e resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
51322025-08-26T21:50:15.351ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2637 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 11105f09-320a-477b-931b-6d15855c6c7e response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
51332025-08-26T21:50:15.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 328 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = beecf95b-9c4a-46a1-9c52-2da88c30d86f response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
51342025-08-26T21:50:16.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 52 reason = Timeout
51352025-08-26T21:50:16.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
51362025-08-26T21:50:16.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
51372025-08-26T21:50:16.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51382025-08-26T21:50:16.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
51392025-08-26T21:50:16.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
51402025-08-26T21:50:16.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
51412025-08-26T21:50:16.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
51422025-08-26T21:50:16.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.544357ms iteration = 52 status = {"chicken_switches_updated":false}
51432025-08-26T21:50:21.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 53 reason = Timeout
51442025-08-26T21:50:21.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
51452025-08-26T21:50:21.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
51462025-08-26T21:50:21.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51472025-08-26T21:50:21.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
51482025-08-26T21:50:21.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
51492025-08-26T21:50:21.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
51502025-08-26T21:50:21.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
51512025-08-26T21:50:21.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 5.805201ms iteration = 53 status = {"chicken_switches_updated":false}
51522025-08-26T21:50:21.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 27 reason = Timeout
51532025-08-26T21:50:21.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51542025-08-26T21:50:21.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51552025-08-26T21:50:21.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51562025-08-26T21:50:21.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
51572025-08-26T21:50:21.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51582025-08-26T21:50:21.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51592025-08-26T21:50:21.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51602025-08-26T21:50:21.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51612025-08-26T21:50:21.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51622025-08-26T21:50:21.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
51632025-08-26T21:50:21.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51642025-08-26T21:50:21.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
51652025-08-26T21:50:21.687ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 27 reason = Timeout
51662025-08-26T21:50:21.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51672025-08-26T21:50:21.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51682025-08-26T21:50:21.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51692025-08-26T21:50:21.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51702025-08-26T21:50:21.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.441177ms iteration = 27 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
51712025-08-26T21:50:21.705ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
51722025-08-26T21:50:21.705ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 30.12482ms iteration = 27 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
51732025-08-26T21:50:25.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 340 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 9ccda85e-d174-46c8-bacb-7d2d0ee8b9e0 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
51742025-08-26T21:50:26.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 54 reason = Timeout
51752025-08-26T21:50:26.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
51762025-08-26T21:50:26.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
51772025-08-26T21:50:26.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51782025-08-26T21:50:26.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
51792025-08-26T21:50:26.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
51802025-08-26T21:50:26.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
51812025-08-26T21:50:26.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
51822025-08-26T21:50:26.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.742884ms iteration = 54 status = {"chicken_switches_updated":false}
51832025-08-26T21:50:31.665ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 10 reason = Timeout
51842025-08-26T21:50:31.665ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 10 reason = Timeout
51852025-08-26T21:50:31.665ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 10 reason = Timeout
51862025-08-26T21:50:31.665ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
51872025-08-26T21:50:31.665ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
51882025-08-26T21:50:31.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
51892025-08-26T21:50:31.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
51902025-08-26T21:50:31.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 55 reason = Timeout
51912025-08-26T21:50:31.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
51922025-08-26T21:50:31.667ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.324713ms iteration = 10 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
51932025-08-26T21:50:31.667ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1135, tv_nsec: 525426847 } })
51942025-08-26T21:50:31.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
51952025-08-26T21:50:31.667ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1135, tv_nsec: 525658583 } })
51962025-08-26T21:50:31.667ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
51972025-08-26T21:50:31.668ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
51982025-08-26T21:50:31.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51992025-08-26T21:50:31.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
52002025-08-26T21:50:31.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
52012025-08-26T21:50:31.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
52022025-08-26T21:50:31.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
52032025-08-26T21:50:31.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.752964ms iteration = 55 status = {"chicken_switches_updated":false}
52042025-08-26T21:50:31.674ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 10 reason = Timeout
52052025-08-26T21:50:31.674ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 10 reason = Timeout
52062025-08-26T21:50:31.674ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 10 reason = Timeout
52072025-08-26T21:50:31.674ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
52082025-08-26T21:50:31.674ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 10 reason = Timeout
52092025-08-26T21:50:31.674ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
52102025-08-26T21:50:31.675ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52112025-08-26T21:50:31.675ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
52122025-08-26T21:50:31.675ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 11 reason = Timeout
52132025-08-26T21:50:31.675ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 10 reason = Timeout
52142025-08-26T21:50:31.675ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
52152025-08-26T21:50:31.675ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
52162025-08-26T21:50:31.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 28 reason = Timeout
52172025-08-26T21:50:31.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52182025-08-26T21:50:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
52192025-08-26T21:50:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52202025-08-26T21:50:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
52212025-08-26T21:50:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
52222025-08-26T21:50:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
52232025-08-26T21:50:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
52242025-08-26T21:50:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
52252025-08-26T21:50:31.676ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52262025-08-26T21:50:31.676ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52272025-08-26T21:50:31.677ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52282025-08-26T21:50:31.677ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52292025-08-26T21:50:31.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52302025-08-26T21:50:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
52312025-08-26T21:50:31.680ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52322025-08-26T21:50:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52332025-08-26T21:50:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
52342025-08-26T21:50:31.681ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
52352025-08-26T21:50:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
52362025-08-26T21:50:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
52372025-08-26T21:50:31.681ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
52382025-08-26T21:50:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
52392025-08-26T21:50:31.681ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
52402025-08-26T21:50:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
52412025-08-26T21:50:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
52422025-08-26T21:50:31.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
52432025-08-26T21:50:31.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52442025-08-26T21:50:31.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
52452025-08-26T21:50:31.683ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
52462025-08-26T21:50:31.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
52472025-08-26T21:50:31.683ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
52482025-08-26T21:50:31.683ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1135, tv_nsec: 541599072 } })
52492025-08-26T21:50:31.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
52502025-08-26T21:50:31.684ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
52512025-08-26T21:50:31.684ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
52522025-08-26T21:50:31.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
52532025-08-26T21:50:31.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52542025-08-26T21:50:31.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
52552025-08-26T21:50:31.685ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52562025-08-26T21:50:31.685ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52572025-08-26T21:50:31.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52582025-08-26T21:50:31.687ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 10 reason = Timeout
52592025-08-26T21:50:31.687ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 10 reason = Timeout
52602025-08-26T21:50:31.687ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
52612025-08-26T21:50:31.687ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
52622025-08-26T21:50:31.687ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
52632025-08-26T21:50:31.687ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 28 reason = Timeout
52642025-08-26T21:50:31.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
52652025-08-26T21:50:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
52662025-08-26T21:50:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
52672025-08-26T21:50:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
52682025-08-26T21:50:31.688ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 10 reason = Timeout
52692025-08-26T21:50:31.688ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 10 reason = Timeout
52702025-08-26T21:50:31.688ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 12.141\u{b5}s iteration = 10 status = {"disabled":true,"errors":[],"sps":[]}
52712025-08-26T21:50:31.688ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
52722025-08-26T21:50:31.688ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 10 reason = Timeout
52732025-08-26T21:50:31.688ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 10 reason = Timeout
52742025-08-26T21:50:31.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
52752025-08-26T21:50:31.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
52762025-08-26T21:50:31.688ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 10 reason = Timeout
52772025-08-26T21:50:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
52782025-08-26T21:50:31.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
52792025-08-26T21:50:31.688ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
52802025-08-26T21:50:31.688ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 10 reason = Timeout
52812025-08-26T21:50:31.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
52822025-08-26T21:50:31.688ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 10 reason = Timeout
52832025-08-26T21:50:31.688ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
52842025-08-26T21:50:31.688ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
52852025-08-26T21:50:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
52862025-08-26T21:50:31.688ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
52872025-08-26T21:50:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
52882025-08-26T21:50:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
52892025-08-26T21:50:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
52902025-08-26T21:50:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
52912025-08-26T21:50:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
52922025-08-26T21:50:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
52932025-08-26T21:50:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
52942025-08-26T21:50:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
52952025-08-26T21:50:31.689ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52962025-08-26T21:50:31.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 2.003075ms iteration = 10 status = {"errors":[],"finish_invoked_ok":[]}
52972025-08-26T21:50:31.690ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
52982025-08-26T21:50:31.690ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
52992025-08-26T21:50:31.690ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
53002025-08-26T21:50:31.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
53012025-08-26T21:50:31.691ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
53022025-08-26T21:50:31.691ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53032025-08-26T21:50:31.691ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
53042025-08-26T21:50:31.691ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 2.860084ms iteration = 10 status = {"errors":[],"garbage_collect_requested":[]}
53052025-08-26T21:50:31.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
53062025-08-26T21:50:31.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
53072025-08-26T21:50:31.693ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
53082025-08-26T21:50:31.693ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
53092025-08-26T21:50:31.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
53102025-08-26T21:50:31.693ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
53112025-08-26T21:50:31.693ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
53122025-08-26T21:50:31.693ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
53132025-08-26T21:50:31.693ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
53142025-08-26T21:50:31.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
53152025-08-26T21:50:31.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
53162025-08-26T21:50:31.694ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
53172025-08-26T21:50:31.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
53182025-08-26T21:50:31.695ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 6.504446ms iteration = 10 status = {"errors":[],"requests_created_ok":[]}
53192025-08-26T21:50:31.696ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 7.381767ms iteration = 10 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
53202025-08-26T21:50:31.696ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
53212025-08-26T21:50:31.696ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
53222025-08-26T21:50:31.696ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 9.487023ms iteration = 10 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
53232025-08-26T21:50:31.707ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
53242025-08-26T21:50:31.707ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 20.658136ms iteration = 10 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
53252025-08-26T21:50:31.711ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
53262025-08-26T21:50:31.711ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 36.718107ms iteration = 28 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
53272025-08-26T21:50:31.745ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 57.009025ms iteration = 10 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
53282025-08-26T21:50:31.751ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 74.177003ms iteration = 11 status = {"physical_disks_added":0}
53292025-08-26T21:50:31.751ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
53302025-08-26T21:50:31.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
53312025-08-26T21:50:31.754ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 79.208036ms iteration = 10 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
53322025-08-26T21:50:31.757ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
53332025-08-26T21:50:31.757ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 82.10248ms iteration = 10 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
53342025-08-26T21:50:31.757ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
53352025-08-26T21:50:31.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
53362025-08-26T21:50:31.758ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53372025-08-26T21:50:31.758ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 71.737357ms iteration = 28 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
53382025-08-26T21:50:31.761ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
53392025-08-26T21:50:31.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
53402025-08-26T21:50:31.761ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
53412025-08-26T21:50:31.762ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53422025-08-26T21:50:31.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
53432025-08-26T21:50:31.763ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
53442025-08-26T21:50:31.763ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53452025-08-26T21:50:31.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
53462025-08-26T21:50:31.766ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53472025-08-26T21:50:31.766ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
53482025-08-26T21:50:31.766ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53492025-08-26T21:50:31.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
53502025-08-26T21:50:31.768ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
53512025-08-26T21:50:31.768ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
53522025-08-26T21:50:31.768ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "898dd736-a6a4-4a39-ae74-f4d12a264cc8", "content-length": "26", "date": "Tue, 26 Aug 2025 21:50:31 GMT"} })
53532025-08-26T21:50:31.768ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
53542025-08-26T21:50:31.769ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
53552025-08-26T21:50:31.769ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
53562025-08-26T21:50:31.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
53572025-08-26T21:50:31.770ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
53582025-08-26T21:50:31.770ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 95.560644ms iteration = 10 status = 0
53592025-08-26T21:50:31.830ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
53602025-08-26T21:50:31.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
53612025-08-26T21:50:31.841ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
53622025-08-26T21:50:31.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
53632025-08-26T21:50:31.843ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53642025-08-26T21:50:31.845ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
53652025-08-26T21:50:31.845ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
53662025-08-26T21:50:31.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
53672025-08-26T21:50:31.847ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53682025-08-26T21:50:31.849ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
53692025-08-26T21:50:31.849ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
53702025-08-26T21:50:31.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
53712025-08-26T21:50:31.865ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
53722025-08-26T21:50:31.865ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
53732025-08-26T21:50:31.866ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b92a5fb-ab32-4520-a7b5-3b7be3d0b60f", "content-length": "26", "date": "Tue, 26 Aug 2025 21:50:31 GMT"} })
53742025-08-26T21:50:31.866ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
53752025-08-26T21:50:31.868ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
53762025-08-26T21:50:31.882ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
53772025-08-26T21:50:31.882ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
53782025-08-26T21:50:31.882ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e2d86d0-bcf3-470c-93f8-9b5edfcb4020", "content-length": "26", "date": "Tue, 26 Aug 2025 21:50:31 GMT"} })
53792025-08-26T21:50:31.882ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
53802025-08-26T21:50:31.882ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
53812025-08-26T21:50:31.882ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
53822025-08-26T21:50:31.883ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ca714dc-8620-464e-adb8-0e9e44615d11", "content-length": "372", "date": "Tue, 26 Aug 2025 21:50:31 GMT"} })
53832025-08-26T21:50:31.883ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53842025-08-26T21:50:31.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
53852025-08-26T21:50:31.884ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
53862025-08-26T21:50:31.886ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
53872025-08-26T21:50:31.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
53882025-08-26T21:50:31.888ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 223.611451ms iteration = 10 status = null
53892025-08-26T21:50:31.897ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
53902025-08-26T21:50:31.897ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
53912025-08-26T21:50:31.897ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "232fa7ef-f375-4628-b2f2-d6e9a70e2871", "date": "Tue, 26 Aug 2025 21:50:31 GMT"} })
53922025-08-26T21:50:31.897ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53932025-08-26T21:50:31.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
53942025-08-26T21:50:31.900ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53952025-08-26T21:50:31.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
53962025-08-26T21:50:31.903ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53972025-08-26T21:50:31.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
53982025-08-26T21:50:31.906ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
53992025-08-26T21:50:31.906ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
54002025-08-26T21:50:31.906ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "9cf4e107-a7af-4d70-bebb-7ca31d461b31", "date": "Tue, 26 Aug 2025 21:50:31 GMT"} })
54012025-08-26T21:50:31.908ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 219.870386ms iteration = 10 status = {}
54022025-08-26T21:50:31.969ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 304.229618ms iteration = 10 status = {}
54032025-08-26T21:50:32.004ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
54042025-08-26T21:50:32.005ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "0961fa4b-3d6e-4e79-90f8-97a68e99d8d2", "content-length": "691", "date": "Tue, 26 Aug 2025 21:50:31 GMT"} })
54052025-08-26T21:50:32.005ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
54062025-08-26T21:50:32.005ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
54072025-08-26T21:50:32.005ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
54082025-08-26T21:50:32.006ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 331.748708ms iteration = 10 status = {}
54092025-08-26T21:50:32.078ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
54102025-08-26T21:50:32.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54112025-08-26T21:50:32.080ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
54122025-08-26T21:50:32.080ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
54132025-08-26T21:50:32.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54142025-08-26T21:50:32.091ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
54152025-08-26T21:50:32.091ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
54162025-08-26T21:50:32.091ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
54172025-08-26T21:50:32.091ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
54182025-08-26T21:50:32.091ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
54192025-08-26T21:50:32.091ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
54202025-08-26T21:50:32.091ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
54212025-08-26T21:50:32.091ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
54222025-08-26T21:50:32.091ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
54232025-08-26T21:50:32.091ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
54242025-08-26T21:50:32.091ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
54252025-08-26T21:50:32.091ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
54262025-08-26T21:50:32.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54272025-08-26T21:50:32.094ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
54282025-08-26T21:50:32.094ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
54292025-08-26T21:50:32.191ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "47cbeeba-219a-4f4c-bace-e6a35a985f84", "date": "Tue, 26 Aug 2025 21:50:31 GMT"} })
54302025-08-26T21:50:32.191ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
54312025-08-26T21:50:32.191ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
54322025-08-26T21:50:32.191ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "80a50d00-e07c-4300-8a9e-4d31c9bfe1ed", "content-length": "682", "date": "Tue, 26 Aug 2025 21:50:31 GMT"} })
54332025-08-26T21:50:32.191ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
54342025-08-26T21:50:32.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54352025-08-26T21:50:32.195ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
54362025-08-26T21:50:32.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54372025-08-26T21:50:32.199ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
54382025-08-26T21:50:32.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54392025-08-26T21:50:32.201ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
54402025-08-26T21:50:32.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54412025-08-26T21:50:32.208ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 533.929186ms iteration = 10 status = {}
54422025-08-26T21:50:35.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 406 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 22a7d657-12fe-4d14-8a6c-e9f152683a6a response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
54432025-08-26T21:50:36.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 56 reason = Timeout
54442025-08-26T21:50:36.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
54452025-08-26T21:50:36.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
54462025-08-26T21:50:36.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54472025-08-26T21:50:36.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
54482025-08-26T21:50:36.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
54492025-08-26T21:50:36.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
54502025-08-26T21:50:36.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
54512025-08-26T21:50:36.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.456065ms iteration = 56 status = {"chicken_switches_updated":false}
54522025-08-26T21:50:41.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 57 reason = Timeout
54532025-08-26T21:50:41.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
54542025-08-26T21:50:41.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
54552025-08-26T21:50:41.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54562025-08-26T21:50:41.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
54572025-08-26T21:50:41.673ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
54582025-08-26T21:50:41.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
54592025-08-26T21:50:41.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 29 reason = Timeout
54602025-08-26T21:50:41.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54612025-08-26T21:50:41.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54622025-08-26T21:50:41.675ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
54632025-08-26T21:50:41.675ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.840877ms iteration = 57 status = {"chicken_switches_updated":false}
54642025-08-26T21:50:41.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54652025-08-26T21:50:41.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
54662025-08-26T21:50:41.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54672025-08-26T21:50:41.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54682025-08-26T21:50:41.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54692025-08-26T21:50:41.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54702025-08-26T21:50:41.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54712025-08-26T21:50:41.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
54722025-08-26T21:50:41.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
54732025-08-26T21:50:41.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
54742025-08-26T21:50:41.687ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 29 reason = Timeout
54752025-08-26T21:50:41.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
54762025-08-26T21:50:41.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
54772025-08-26T21:50:41.689ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
54782025-08-26T21:50:41.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
54792025-08-26T21:50:41.691ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.764475ms iteration = 29 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
54802025-08-26T21:50:41.701ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
54812025-08-26T21:50:41.701ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.16142ms iteration = 29 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
54822025-08-26T21:50:45.346ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 3992f70c-8f25-4008-8a88-ea4bb9503b40 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
54832025-08-26T21:50:45.346ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 3992f70c-8f25-4008-8a88-ea4bb9503b40 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
54842025-08-26T21:50:45.349ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2681 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 3992f70c-8f25-4008-8a88-ea4bb9503b40 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
54852025-08-26T21:50:45.349ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 1130f0c1-86d1-41fd-9ba0-2369e96436cd roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
54862025-08-26T21:50:45.349ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 1130f0c1-86d1-41fd-9ba0-2369e96436cd resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
54872025-08-26T21:50:45.351ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2467 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 1130f0c1-86d1-41fd-9ba0-2369e96436cd response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
54882025-08-26T21:50:45.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 324 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 645c0aba-9db7-47af-83d1-3453f0f75ffc response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
54892025-08-26T21:50:46.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 58 reason = Timeout
54902025-08-26T21:50:46.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
54912025-08-26T21:50:46.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
54922025-08-26T21:50:46.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54932025-08-26T21:50:46.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
54942025-08-26T21:50:46.673ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
54952025-08-26T21:50:46.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
54962025-08-26T21:50:46.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
54972025-08-26T21:50:46.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.06662ms iteration = 58 status = {"chicken_switches_updated":false}
54982025-08-26T21:50:51.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 59 reason = Timeout
54992025-08-26T21:50:51.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
55002025-08-26T21:50:51.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
55012025-08-26T21:50:51.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55022025-08-26T21:50:51.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
55032025-08-26T21:50:51.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
55042025-08-26T21:50:51.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
55052025-08-26T21:50:51.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
55062025-08-26T21:50:51.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.99174ms iteration = 59 status = {"chicken_switches_updated":false}
55072025-08-26T21:50:51.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 30 reason = Timeout
55082025-08-26T21:50:51.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55092025-08-26T21:50:51.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
55102025-08-26T21:50:51.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55112025-08-26T21:50:51.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
55122025-08-26T21:50:51.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55132025-08-26T21:50:51.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
55142025-08-26T21:50:51.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55152025-08-26T21:50:51.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
55162025-08-26T21:50:51.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55172025-08-26T21:50:51.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
55182025-08-26T21:50:51.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55192025-08-26T21:50:51.687ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 30 reason = Timeout
55202025-08-26T21:50:51.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
55212025-08-26T21:50:51.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
55222025-08-26T21:50:51.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
55232025-08-26T21:50:51.689ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
55242025-08-26T21:50:51.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
55252025-08-26T21:50:51.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.459127ms iteration = 30 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
55262025-08-26T21:50:51.700ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
55272025-08-26T21:50:51.700ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.446293ms iteration = 30 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
55282025-08-26T21:50:55.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 615 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = e226a700-69f2-4644-8866-a9fb01448e3c response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
55292025-08-26T21:50:56.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 60 reason = Timeout
55302025-08-26T21:50:56.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
55312025-08-26T21:50:56.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
55322025-08-26T21:50:56.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55332025-08-26T21:50:56.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
55342025-08-26T21:50:56.673ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
55352025-08-26T21:50:56.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
55362025-08-26T21:50:56.675ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
55372025-08-26T21:50:56.675ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.039731ms iteration = 60 status = {"chicken_switches_updated":false}
55382025-08-26T21:51:00.346ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 165c100e-2aa1-4228-8e97-560ec98e325d roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
55392025-08-26T21:51:00.346ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 165c100e-2aa1-4228-8e97-560ec98e325d resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
55402025-08-26T21:51:00.350ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3986 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 165c100e-2aa1-4228-8e97-560ec98e325d response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
55412025-08-26T21:51:00.350ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 30d4baa9-3736-45fd-951e-dbb2e3cf86e8 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
55422025-08-26T21:51:00.351ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 30d4baa9-3736-45fd-951e-dbb2e3cf86e8 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
55432025-08-26T21:51:00.353ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2824 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 30d4baa9-3736-45fd-951e-dbb2e3cf86e8 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
55442025-08-26T21:51:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 6 reason = Timeout
55452025-08-26T21:51:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 6 reason = Timeout
55462025-08-26T21:51:01.665ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 11 reason = Timeout
55472025-08-26T21:51:01.665ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
55482025-08-26T21:51:01.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
55492025-08-26T21:51:01.665ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 11 reason = Timeout
55502025-08-26T21:51:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 8 reason = Timeout
55512025-08-26T21:51:01.665ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
55522025-08-26T21:51:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 8 reason = Timeout
55532025-08-26T21:51:01.665ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 11 reason = Timeout
55542025-08-26T21:51:01.665ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 6 reason = Timeout
55552025-08-26T21:51:01.665ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:41:01.665839373 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
55562025-08-26T21:51:01.665ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
55572025-08-26T21:51:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 6 reason = Timeout
55582025-08-26T21:51:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 6 reason = Timeout
55592025-08-26T21:51:01.665ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
55602025-08-26T21:51:01.665ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
55612025-08-26T21:51:01.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
55622025-08-26T21:51:01.665ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 6 reason = Timeout
55632025-08-26T21:51:01.665ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
55642025-08-26T21:51:01.665ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 7 reason = Timeout
55652025-08-26T21:51:01.665ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
55662025-08-26T21:51:01.665ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 15.79\u{b5}s iteration = 7 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
55672025-08-26T21:51:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
55682025-08-26T21:51:01.666ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1165, tv_nsec: 524074267 } })
55692025-08-26T21:51:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
55702025-08-26T21:51:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
55712025-08-26T21:51:01.666ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1165, tv_nsec: 524256991 } })
55722025-08-26T21:51:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
55732025-08-26T21:51:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
55742025-08-26T21:51:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
55752025-08-26T21:51:01.666ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
55762025-08-26T21:51:01.666ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 933.631\u{b5}s iteration = 6 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
55772025-08-26T21:51:01.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 61 reason = Timeout
55782025-08-26T21:51:01.666ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
55792025-08-26T21:51:01.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
55802025-08-26T21:51:01.666ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
55812025-08-26T21:51:01.666ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
55822025-08-26T21:51:01.666ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.120855ms iteration = 6 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
55832025-08-26T21:51:01.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
55842025-08-26T21:51:01.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
55852025-08-26T21:51:01.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
55862025-08-26T21:51:01.668ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1165, tv_nsec: 526240656 } })
55872025-08-26T21:51:01.668ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55882025-08-26T21:51:01.668ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
55892025-08-26T21:51:01.668ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:41:01.665839373 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
55902025-08-26T21:51:01.668ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 2.921276ms iteration = 6 status = {"expiration":"2025-08-26T21:41:01.665839373Z","pruned":[]}
55912025-08-26T21:51:01.668ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55922025-08-26T21:51:01.668ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1165, tv_nsec: 526785968 } })
55932025-08-26T21:51:01.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55942025-08-26T21:51:01.669ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55952025-08-26T21:51:01.669ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
55962025-08-26T21:51:01.669ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.239526ms iteration = 11 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
55972025-08-26T21:51:01.671ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
55982025-08-26T21:51:01.671ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
55992025-08-26T21:51:01.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
56002025-08-26T21:51:01.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
56012025-08-26T21:51:01.673ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
56022025-08-26T21:51:01.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
56032025-08-26T21:51:01.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 8 reason = Timeout
56042025-08-26T21:51:01.674ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
56052025-08-26T21:51:01.674ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 10.061\u{b5}s iteration = 8 status = "Disabled"
56062025-08-26T21:51:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
56072025-08-26T21:51:01.674ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
56082025-08-26T21:51:01.674ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
56092025-08-26T21:51:01.674ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
56102025-08-26T21:51:01.674ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
56112025-08-26T21:51:01.674ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
56122025-08-26T21:51:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
56132025-08-26T21:51:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
56142025-08-26T21:51:01.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 3 reason = Timeout
56152025-08-26T21:51:01.675ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 11 reason = Timeout
56162025-08-26T21:51:01.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
56172025-08-26T21:51:01.675ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
56182025-08-26T21:51:01.675ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 11 reason = Timeout
56192025-08-26T21:51:01.675ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
56202025-08-26T21:51:01.675ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 11 reason = Timeout
56212025-08-26T21:51:01.675ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56222025-08-26T21:51:01.675ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
56232025-08-26T21:51:01.675ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 12 reason = Timeout
56242025-08-26T21:51:01.675ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 11 reason = Timeout
56252025-08-26T21:51:01.675ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
56262025-08-26T21:51:01.675ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
56272025-08-26T21:51:01.675ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 6 reason = Timeout
56282025-08-26T21:51:01.675ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 11 reason = Timeout
56292025-08-26T21:51:01.675ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
56302025-08-26T21:51:01.675ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
56312025-08-26T21:51:01.675ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
56322025-08-26T21:51:01.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 31 reason = Timeout
56332025-08-26T21:51:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
56342025-08-26T21:51:01.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56352025-08-26T21:51:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
56362025-08-26T21:51:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
56372025-08-26T21:51:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
56382025-08-26T21:51:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
56392025-08-26T21:51:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
56402025-08-26T21:51:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56412025-08-26T21:51:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
56422025-08-26T21:51:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
56432025-08-26T21:51:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
56442025-08-26T21:51:01.676ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56452025-08-26T21:51:01.676ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
56462025-08-26T21:51:01.676ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 9.963025ms iteration = 61 status = {"chicken_switches_updated":false}
56472025-08-26T21:51:01.677ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
56482025-08-26T21:51:01.677ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
56492025-08-26T21:51:01.677ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
56502025-08-26T21:51:01.677ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
56512025-08-26T21:51:01.677ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56522025-08-26T21:51:01.677ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56532025-08-26T21:51:01.677ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56542025-08-26T21:51:01.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
56552025-08-26T21:51:01.677ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56562025-08-26T21:51:01.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56572025-08-26T21:51:01.678ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56582025-08-26T21:51:01.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56592025-08-26T21:51:01.679ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56602025-08-26T21:51:01.681ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
56612025-08-26T21:51:01.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
56622025-08-26T21:51:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
56632025-08-26T21:51:01.682ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
56642025-08-26T21:51:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
56652025-08-26T21:51:01.683ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 7.893458ms iteration = 11 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
56662025-08-26T21:51:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
56672025-08-26T21:51:01.683ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
56682025-08-26T21:51:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
56692025-08-26T21:51:01.683ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
56702025-08-26T21:51:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
56712025-08-26T21:51:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
56722025-08-26T21:51:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
56732025-08-26T21:51:01.684ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
56742025-08-26T21:51:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
56752025-08-26T21:51:01.684ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
56762025-08-26T21:51:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
56772025-08-26T21:51:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
56782025-08-26T21:51:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
56792025-08-26T21:51:01.684ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
56802025-08-26T21:51:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
56812025-08-26T21:51:01.684ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
56822025-08-26T21:51:01.684ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
56832025-08-26T21:51:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
56842025-08-26T21:51:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
56852025-08-26T21:51:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
56862025-08-26T21:51:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
56872025-08-26T21:51:01.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
56882025-08-26T21:51:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
56892025-08-26T21:51:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
56902025-08-26T21:51:01.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56912025-08-26T21:51:01.685ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
56922025-08-26T21:51:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
56932025-08-26T21:51:01.685ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
56942025-08-26T21:51:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
56952025-08-26T21:51:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
56962025-08-26T21:51:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
56972025-08-26T21:51:01.686ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
56982025-08-26T21:51:01.686ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
56992025-08-26T21:51:01.686ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
57002025-08-26T21:51:01.686ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57012025-08-26T21:51:01.686ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 2 reason = Timeout
57022025-08-26T21:51:01.686ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
57032025-08-26T21:51:01.686ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
57042025-08-26T21:51:01.686ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
57052025-08-26T21:51:01.686ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 11 reason = Timeout
57062025-08-26T21:51:01.686ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
57072025-08-26T21:51:01.686ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 6 reason = Timeout
57082025-08-26T21:51:01.686ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
57092025-08-26T21:51:01.686ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 11 reason = Timeout
57102025-08-26T21:51:01.686ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
57112025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
57122025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 31 reason = Timeout
57132025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
57142025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
57152025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
57162025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
57172025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
57182025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
57192025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57202025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
57212025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
57222025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
57232025-08-26T21:51:01.687ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1165, tv_nsec: 545274285 } })
57242025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
57252025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
57262025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
57272025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 0 version = 2
57282025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
57292025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57302025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 2 reason = Timeout
57312025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 11 reason = Timeout
57322025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
57332025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 6 reason = Timeout
57342025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
57352025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
57362025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 11 reason = Timeout
57372025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
57382025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 6 reason = Timeout
57392025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 11 reason = Timeout
57402025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 11 reason = Timeout
57412025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
57422025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
57432025-08-26T21:51:01.687ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
57442025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57452025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 11 reason = Timeout
57462025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
57472025-08-26T21:51:01.687ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 6 reason = Timeout
57482025-08-26T21:51:01.688ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 11 reason = Timeout
57492025-08-26T21:51:01.688ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
57502025-08-26T21:51:01.688ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 6 reason = Timeout
57512025-08-26T21:51:01.688ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 11 reason = Timeout
57522025-08-26T21:51:01.688ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 18.511\u{b5}s iteration = 11 status = {"disabled":true,"errors":[],"sps":[]}
57532025-08-26T21:51:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
57542025-08-26T21:51:01.688ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
57552025-08-26T21:51:01.688ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
57562025-08-26T21:51:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
57572025-08-26T21:51:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
57582025-08-26T21:51:01.688ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
57592025-08-26T21:51:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
57602025-08-26T21:51:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
57612025-08-26T21:51:01.688ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
57622025-08-26T21:51:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57632025-08-26T21:51:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
57642025-08-26T21:51:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
57652025-08-26T21:51:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
57662025-08-26T21:51:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
57672025-08-26T21:51:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
57682025-08-26T21:51:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
57692025-08-26T21:51:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
57702025-08-26T21:51:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
57712025-08-26T21:51:01.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
57722025-08-26T21:51:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
57732025-08-26T21:51:01.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
57742025-08-26T21:51:01.689ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57752025-08-26T21:51:01.689ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57762025-08-26T21:51:01.689ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57772025-08-26T21:51:01.689ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57782025-08-26T21:51:01.690ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
57792025-08-26T21:51:01.690ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
57802025-08-26T21:51:01.690ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
57812025-08-26T21:51:01.690ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57822025-08-26T21:51:01.690ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
57832025-08-26T21:51:01.690ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57842025-08-26T21:51:01.690ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
57852025-08-26T21:51:01.690ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
57862025-08-26T21:51:01.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
57872025-08-26T21:51:01.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
57882025-08-26T21:51:01.691ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 3.439987ms iteration = 11 status = {"errors":[],"finish_invoked_ok":[]}
57892025-08-26T21:51:01.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
57902025-08-26T21:51:01.691ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 3.886707ms iteration = 6 status = {"errors":[],"found_port_ok":[]}
57912025-08-26T21:51:01.692ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
57922025-08-26T21:51:01.692ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 4.220105ms iteration = 6 status = {"by_rx":{},"error":null}
57932025-08-26T21:51:01.692ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57942025-08-26T21:51:01.693ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
57952025-08-26T21:51:01.693ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 4.88635ms iteration = 11 status = {"errors":[],"garbage_collect_requested":[]}
57962025-08-26T21:51:01.693ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
57972025-08-26T21:51:01.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
57982025-08-26T21:51:01.694ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
57992025-08-26T21:51:01.694ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
58002025-08-26T21:51:01.694ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
58012025-08-26T21:51:01.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
58022025-08-26T21:51:01.694ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58032025-08-26T21:51:01.695ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
58042025-08-26T21:51:01.695ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
58052025-08-26T21:51:01.695ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 7.217543ms iteration = 6 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
58062025-08-26T21:51:01.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58072025-08-26T21:51:01.695ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
58082025-08-26T21:51:01.695ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
58092025-08-26T21:51:01.695ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
58102025-08-26T21:51:01.695ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 19.911169ms iteration = 11 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
58112025-08-26T21:51:01.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
58122025-08-26T21:51:01.696ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
58132025-08-26T21:51:01.696ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
58142025-08-26T21:51:01.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
58152025-08-26T21:51:01.696ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
58162025-08-26T21:51:01.696ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
58172025-08-26T21:51:01.696ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
58182025-08-26T21:51:01.697ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
58192025-08-26T21:51:01.697ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58202025-08-26T21:51:01.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
58212025-08-26T21:51:01.697ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
58222025-08-26T21:51:01.697ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
58232025-08-26T21:51:01.697ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
58242025-08-26T21:51:01.697ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
58252025-08-26T21:51:01.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
58262025-08-26T21:51:01.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58272025-08-26T21:51:01.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
58282025-08-26T21:51:01.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
58292025-08-26T21:51:01.697ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
58302025-08-26T21:51:01.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
58312025-08-26T21:51:01.699ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
58322025-08-26T21:51:01.699ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58332025-08-26T21:51:01.699ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58342025-08-26T21:51:01.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
58352025-08-26T21:51:01.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58362025-08-26T21:51:01.699ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58372025-08-26T21:51:01.699ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58382025-08-26T21:51:01.699ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 4 version = 2
58392025-08-26T21:51:01.699ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
58402025-08-26T21:51:01.699ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-08-26 21:46:05.296205 UTC dns_group = external generation = 2
58412025-08-26T21:51:01.699ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 34.075718ms iteration = 6 status = {"generation":2}
58422025-08-26T21:51:01.700ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
58432025-08-26T21:51:01.700ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
58442025-08-26T21:51:01.700ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-08-26 21:46:04.065217 UTC dns_group = internal generation = 1
58452025-08-26T21:51:01.700ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 35.137012ms iteration = 6 status = {"generation":1}
58462025-08-26T21:51:01.701ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 13.675459ms iteration = 6 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
58472025-08-26T21:51:01.701ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 13.934464ms iteration = 11 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
58482025-08-26T21:51:01.701ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58492025-08-26T21:51:01.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58502025-08-26T21:51:01.702ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
58512025-08-26T21:51:01.702ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58522025-08-26T21:51:01.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58532025-08-26T21:51:01.703ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
58542025-08-26T21:51:01.703ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
58552025-08-26T21:51:01.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
58562025-08-26T21:51:01.704ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58572025-08-26T21:51:01.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58582025-08-26T21:51:01.705ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 18.20749ms iteration = 31 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
58592025-08-26T21:51:01.706ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
58602025-08-26T21:51:01.706ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 19.440858ms iteration = 11 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
58612025-08-26T21:51:01.706ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58622025-08-26T21:51:01.709ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-08-26T21:45:59.866217Z, time_modified: 2025-08-26T21:45:59.866217Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }) }
58632025-08-26T21:51:01.709ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 43.729756ms iteration = 6 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]}},"default_endpoint":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
58642025-08-26T21:51:01.713ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
58652025-08-26T21:51:01.713ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58662025-08-26T21:51:01.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58672025-08-26T21:51:01.721ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58682025-08-26T21:51:01.724ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58692025-08-26T21:51:01.728ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58702025-08-26T21:51:01.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58712025-08-26T21:51:01.731ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58722025-08-26T21:51:01.738ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
58732025-08-26T21:51:01.738ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58742025-08-26T21:51:01.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58752025-08-26T21:51:01.745ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
58762025-08-26T21:51:01.745ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 69.459176ms iteration = 31 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
58772025-08-26T21:51:01.749ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58782025-08-26T21:51:01.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58792025-08-26T21:51:01.756ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58802025-08-26T21:51:01.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58812025-08-26T21:51:01.761ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58822025-08-26T21:51:01.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58832025-08-26T21:51:01.768ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58842025-08-26T21:51:01.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58852025-08-26T21:51:01.769ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
58862025-08-26T21:51:01.769ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
58872025-08-26T21:51:01.769ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 94.278755ms iteration = 6 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
58882025-08-26T21:51:01.771ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58892025-08-26T21:51:01.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58902025-08-26T21:51:01.774ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
58912025-08-26T21:51:01.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
58922025-08-26T21:51:01.775ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
58932025-08-26T21:51:01.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
58942025-08-26T21:51:01.775ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
58952025-08-26T21:51:01.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
58962025-08-26T21:51:01.777ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58972025-08-26T21:51:01.777ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
58982025-08-26T21:51:01.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
58992025-08-26T21:51:01.778ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59002025-08-26T21:51:01.780ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59012025-08-26T21:51:01.782ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
59022025-08-26T21:51:01.782ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
59032025-08-26T21:51:01.782ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 95.419421ms iteration = 11 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
59042025-08-26T21:51:01.782ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
59052025-08-26T21:51:01.783ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
59062025-08-26T21:51:01.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
59072025-08-26T21:51:01.784ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
59082025-08-26T21:51:01.785ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59092025-08-26T21:51:01.785ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
59102025-08-26T21:51:01.785ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
59112025-08-26T21:51:01.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
59122025-08-26T21:51:01.788ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
59132025-08-26T21:51:01.788ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
59142025-08-26T21:51:01.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
59152025-08-26T21:51:01.789ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
59162025-08-26T21:51:01.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
59172025-08-26T21:51:01.791ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 116.118767ms iteration = 11 status = 0
59182025-08-26T21:51:01.795ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
59192025-08-26T21:51:01.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
59202025-08-26T21:51:01.796ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
59212025-08-26T21:51:01.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
59222025-08-26T21:51:01.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59232025-08-26T21:51:01.798ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
59242025-08-26T21:51:01.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
59252025-08-26T21:51:01.800ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
59262025-08-26T21:51:01.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
59272025-08-26T21:51:01.801ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 113.804445ms iteration = 11 status = {"errors":[],"requests_created_ok":[]}
59282025-08-26T21:51:01.802ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
59292025-08-26T21:51:01.802ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
59302025-08-26T21:51:01.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
59312025-08-26T21:51:01.803ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
59322025-08-26T21:51:01.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
59332025-08-26T21:51:01.803ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59342025-08-26T21:51:01.807ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
59352025-08-26T21:51:01.807ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
59362025-08-26T21:51:01.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
59372025-08-26T21:51:01.808ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
59382025-08-26T21:51:01.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
59392025-08-26T21:51:01.814ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
59402025-08-26T21:51:01.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
59412025-08-26T21:51:01.816ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
59422025-08-26T21:51:01.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
59432025-08-26T21:51:01.824ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
59442025-08-26T21:51:01.824ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59452025-08-26T21:51:01.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
59462025-08-26T21:51:01.826ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
59472025-08-26T21:51:01.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
59482025-08-26T21:51:01.827ZDEBGnexus (BackgroundTasks): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
59492025-08-26T21:51:01.827ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
59502025-08-26T21:51:01.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
59512025-08-26T21:51:01.829ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59522025-08-26T21:51:01.835ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
59532025-08-26T21:51:01.835ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
59542025-08-26T21:51:01.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
59552025-08-26T21:51:01.837ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
59562025-08-26T21:51:01.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
59572025-08-26T21:51:01.841ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
59582025-08-26T21:51:01.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
59592025-08-26T21:51:01.847ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59602025-08-26T21:51:01.852ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
59612025-08-26T21:51:01.852ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
59622025-08-26T21:51:01.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
59632025-08-26T21:51:01.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
59642025-08-26T21:51:01.865ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
59652025-08-26T21:51:01.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
59662025-08-26T21:51:01.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
59672025-08-26T21:51:01.867ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 179.624329ms iteration = 11 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
59682025-08-26T21:51:01.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59692025-08-26T21:51:01.868ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
59702025-08-26T21:51:01.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
59712025-08-26T21:51:01.869ZDEBGnexus (BackgroundTasks): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
59722025-08-26T21:51:01.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
59732025-08-26T21:51:01.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
59742025-08-26T21:51:01.870ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
59752025-08-26T21:51:01.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
59762025-08-26T21:51:01.872ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
59772025-08-26T21:51:01.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
59782025-08-26T21:51:01.876ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
59792025-08-26T21:51:01.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
59802025-08-26T21:51:01.880ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
59812025-08-26T21:51:01.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
59822025-08-26T21:51:01.881ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
59832025-08-26T21:51:01.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
59842025-08-26T21:51:01.883ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59852025-08-26T21:51:01.884ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
59862025-08-26T21:51:01.884ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
59872025-08-26T21:51:01.884ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa9b6fb0-86b0-4773-a033-85e3a6ca8173", "content-length": "26", "date": "Tue, 26 Aug 2025 21:51:01 GMT"} })
59882025-08-26T21:51:01.884ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
59892025-08-26T21:51:01.886ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
59902025-08-26T21:51:01.888ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
59912025-08-26T21:51:01.888ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
59922025-08-26T21:51:01.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
59932025-08-26T21:51:01.890ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
59942025-08-26T21:51:01.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
59952025-08-26T21:51:01.893ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
59962025-08-26T21:51:01.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
59972025-08-26T21:51:01.895ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59982025-08-26T21:51:01.895ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
59992025-08-26T21:51:01.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
60002025-08-26T21:51:01.899ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
60012025-08-26T21:51:01.899ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
60022025-08-26T21:51:01.899ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
60032025-08-26T21:51:01.900ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 213.123944ms iteration = 6 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
60042025-08-26T21:51:01.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
60052025-08-26T21:51:01.907ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 0106180c-e6f8-4d65-969e-4fbbddf596a1, kind: Service { id: 60abf731-31e1-4111-a3e2-65da89eaed87 }, name: Name("external-dns-60abf731-31e1-4111-a3e2-65da89eaed87"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 81])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b69ae953-8f67-4cb3-9521-db9e707a6f2d, kind: Service { id: 84c849eb-0949-41d9-ad02-9a5509144564 }, name: Name("external-dns-84c849eb-0949-41d9-ad02-9a5509144564"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 136])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 0802405a-a5f2-402c-94da-ccafb58bccb5, kind: Service { id: 4996dcc4-c530-4f64-b407-fa372462bd9e }, name: Name("nexus-4996dcc4-c530-4f64-b407-fa372462bd9e"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 1acb6bfd-558b-4271-bbf5-58e0d089b710, kind: Service { id: 0205eaf0-3fc6-4ad5-b09b-732af233e9c1 }, name: Name("nexus-0205eaf0-3fc6-4ad5-b09b-732af233e9c1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 142])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cbb99fa2-edc6-497d-9c47-8dd2642f8945, kind: Service { id: 5d0959e5-47a6-4bf3-a52b-57950b9d900d }, name: Name("nexus-5d0959e5-47a6-4bf3-a52b-57950b9d900d"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 80])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
60062025-08-26T21:51:01.907ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
60072025-08-26T21:51:01.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
60082025-08-26T21:51:01.909ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60092025-08-26T21:51:01.910ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
60102025-08-26T21:51:01.910ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
60112025-08-26T21:51:01.910ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "31a91285-7579-420e-af4a-8565eae04212", "content-length": "26", "date": "Tue, 26 Aug 2025 21:51:01 GMT"} })
60122025-08-26T21:51:01.910ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
60132025-08-26T21:51:01.912ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
60142025-08-26T21:51:01.912ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
60152025-08-26T21:51:01.912ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
60162025-08-26T21:51:01.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
60172025-08-26T21:51:01.913ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 237.701388ms iteration = 12 status = {"physical_disks_added":0}
60182025-08-26T21:51:01.917ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
60192025-08-26T21:51:01.917ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0106180c-e6f8-4d65-969e-4fbbddf596a1, kind: Service { id: 60abf731-31e1-4111-a3e2-65da89eaed87 }, name: Name("external-dns-60abf731-31e1-4111-a3e2-65da89eaed87"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 81])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b69ae953-8f67-4cb3-9521-db9e707a6f2d, kind: Service { id: 84c849eb-0949-41d9-ad02-9a5509144564 }, name: Name("external-dns-84c849eb-0949-41d9-ad02-9a5509144564"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 136])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0802405a-a5f2-402c-94da-ccafb58bccb5, kind: Service { id: 4996dcc4-c530-4f64-b407-fa372462bd9e }, name: Name("nexus-4996dcc4-c530-4f64-b407-fa372462bd9e"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 1acb6bfd-558b-4271-bbf5-58e0d089b710, kind: Service { id: 0205eaf0-3fc6-4ad5-b09b-732af233e9c1 }, name: Name("nexus-0205eaf0-3fc6-4ad5-b09b-732af233e9c1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 142])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cbb99fa2-edc6-497d-9c47-8dd2642f8945, kind: Service { id: 5d0959e5-47a6-4bf3-a52b-57950b9d900d }, name: Name("nexus-5d0959e5-47a6-4bf3-a52b-57950b9d900d"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 80])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 0106180c-e6f8-4d65-969e-4fbbddf596a1, kind: Service { id: 60abf731-31e1-4111-a3e2-65da89eaed87 }, name: Name("external-dns-60abf731-31e1-4111-a3e2-65da89eaed87"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 81])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b69ae953-8f67-4cb3-9521-db9e707a6f2d, kind: Service { id: 84c849eb-0949-41d9-ad02-9a5509144564 }, name: Name("external-dns-84c849eb-0949-41d9-ad02-9a5509144564"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 136])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0802405a-a5f2-402c-94da-ccafb58bccb5, kind: Service { id: 4996dcc4-c530-4f64-b407-fa372462bd9e }, name: Name("nexus-4996dcc4-c530-4f64-b407-fa372462bd9e"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 1acb6bfd-558b-4271-bbf5-58e0d089b710, kind: Service { id: 0205eaf0-3fc6-4ad5-b09b-732af233e9c1 }, name: Name("nexus-0205eaf0-3fc6-4ad5-b09b-732af233e9c1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 142])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cbb99fa2-edc6-497d-9c47-8dd2642f8945, kind: Service { id: 5d0959e5-47a6-4bf3-a52b-57950b9d900d }, name: Name("nexus-5d0959e5-47a6-4bf3-a52b-57950b9d900d"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 80])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
60202025-08-26T21:51:01.917ZDEBGnexus (service-firewall-rule-progator): resolved 3 rules for sleds background_task = service_firewall_rule_propagation
60212025-08-26T21:51:01.986ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
60222025-08-26T21:51:01.986ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
60232025-08-26T21:51:01.987ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
60242025-08-26T21:51:01.988ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
60252025-08-26T21:51:01.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
60262025-08-26T21:51:01.989ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60272025-08-26T21:51:01.990ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60282025-08-26T21:51:01.991ZDEBGnexus (BackgroundTasks): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
60292025-08-26T21:51:01.991ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
60302025-08-26T21:51:01.992ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
60312025-08-26T21:51:01.994ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
60322025-08-26T21:51:01.994ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
60332025-08-26T21:51:01.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
60342025-08-26T21:51:01.994ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
60352025-08-26T21:51:01.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
60362025-08-26T21:51:01.997ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60372025-08-26T21:51:02.000ZDEBGnexus (BackgroundTasks): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
60382025-08-26T21:51:02.000ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
60392025-08-26T21:51:02.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
60402025-08-26T21:51:02.002ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
60412025-08-26T21:51:02.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
60422025-08-26T21:51:02.004ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60432025-08-26T21:51:02.005ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
60442025-08-26T21:51:02.005ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
60452025-08-26T21:51:02.006ZDEBGnexus (BackgroundTasks): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
60462025-08-26T21:51:02.006ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
60472025-08-26T21:51:02.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
60482025-08-26T21:51:02.006ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60492025-08-26T21:51:02.008ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
60502025-08-26T21:51:02.008ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
60512025-08-26T21:51:02.008ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
60522025-08-26T21:51:02.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
60532025-08-26T21:51:02.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
60542025-08-26T21:51:02.010ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60552025-08-26T21:51:02.014ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = tuf_artifact_replication body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/artifacts-config
60562025-08-26T21:51:02.015ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts-config", status: 204, headers: {"x-request-id": "a63c17c2-d8c7-4134-828c-695603681dec", "date": "Tue, 26 Aug 2025 21:51:02 GMT"} })
60572025-08-26T21:51:02.015ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = tuf_artifact_replication body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/artifacts
60582025-08-26T21:51:02.015ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6c95138-d77b-45de-a13a-d3e7e4379f48", "content-length": "26", "date": "Tue, 26 Aug 2025 21:51:02 GMT"} })
60592025-08-26T21:51:02.015ZDEBGnexus (BackgroundTasks): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
60602025-08-26T21:51:02.015ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
60612025-08-26T21:51:02.016ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
60622025-08-26T21:51:02.017ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
60632025-08-26T21:51:02.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
60642025-08-26T21:51:02.019ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60652025-08-26T21:51:02.020ZINFOnexus (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication file = nexus/src/app/background/tasks/tuf_artifact_replication.rs:672 sled = http://[fd00:1122:3344:101::1]:12345
60662025-08-26T21:51:02.022ZDEBGnexus (BackgroundTasks): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
60672025-08-26T21:51:02.022ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
60682025-08-26T21:51:02.022ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 334.274725ms iteration = 2 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"generation":1,"operation":"put_config","target_sled":"dd0c655a-95c2-48fc-b4c3-9733ed4b0faa","time":"2025-08-26T21:51:02.015071525Z"},{"operation":"list","target_sled":"dd0c655a-95c2-48fc-b4c3-9733ed4b0faa","time":"2025-08-26T21:51:02.020355155Z"}]}
60692025-08-26T21:51:02.022ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
60702025-08-26T21:51:02.024ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
60712025-08-26T21:51:02.024ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
60722025-08-26T21:51:02.026ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60732025-08-26T21:51:02.027ZDEBGnexus (BackgroundTasks): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
60742025-08-26T21:51:02.027ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
60752025-08-26T21:51:02.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
60762025-08-26T21:51:02.027ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, time_created: 2025-08-26T21:46:02.113579Z, time_modified: 2025-08-26T21:46:02.113579Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
60772025-08-26T21:51:02.027ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
60782025-08-26T21:51:02.029ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
60792025-08-26T21:51:02.029ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
60802025-08-26T21:51:02.031ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60812025-08-26T21:51:02.033ZDEBGnexus (BackgroundTasks): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
60822025-08-26T21:51:02.033ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
60832025-08-26T21:51:02.033ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
60842025-08-26T21:51:02.035ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
60852025-08-26T21:51:02.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
60862025-08-26T21:51:02.036ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60872025-08-26T21:51:02.038ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
60882025-08-26T21:51:02.038ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
60892025-08-26T21:51:02.038ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
60902025-08-26T21:51:02.040ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60912025-08-26T21:51:02.042ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
60922025-08-26T21:51:02.043ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
60932025-08-26T21:51:02.043ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
60942025-08-26T21:51:02.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
60952025-08-26T21:51:02.051ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:113 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
60962025-08-26T21:51:02.059ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 376.128379ms iteration = 3 status = {"blueprint_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","inventory_collection_id":"22ce40dd-e0b9-460a-8151-60c0f8e6fa4f","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":8,"num_already_tombstoned":0,"num_inserted":1,"num_not_in_inventory":0,"num_tombstoned":0}}}
60972025-08-26T21:51:02.080ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
60982025-08-26T21:51:02.080ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
60992025-08-26T21:51:02.081ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "21a42291-5473-4d80-9446-6444f9d4163d", "content-length": "372", "date": "Tue, 26 Aug 2025 21:51:02 GMT"} })
61002025-08-26T21:51:02.081ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
61012025-08-26T21:51:02.081ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
61022025-08-26T21:51:02.081ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
61032025-08-26T21:51:02.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
61042025-08-26T21:51:02.081ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "741a46e6-7152-40c4-a602-fae2cf36fbcf", "content-length": "26", "date": "Tue, 26 Aug 2025 21:51:01 GMT"} })
61052025-08-26T21:51:02.081ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
61062025-08-26T21:51:02.083ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
61072025-08-26T21:51:02.095ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
61082025-08-26T21:51:02.095ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
61092025-08-26T21:51:02.096ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "9f2b73c3-0190-4fa1-99d3-f5f9c794b2cf", "date": "Tue, 26 Aug 2025 21:51:02 GMT"} })
61102025-08-26T21:51:02.096ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
61112025-08-26T21:51:02.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
61122025-08-26T21:51:02.098ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
61132025-08-26T21:51:02.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
61142025-08-26T21:51:02.101ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
61152025-08-26T21:51:02.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
61162025-08-26T21:51:02.104ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
61172025-08-26T21:51:02.104ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
61182025-08-26T21:51:02.104ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "cd4a77bc-71c7-464d-a312-c2f02badf7e1", "date": "Tue, 26 Aug 2025 21:51:02 GMT"} })
61192025-08-26T21:51:02.105ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 418.114285ms iteration = 11 status = {}
61202025-08-26T21:51:02.123ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
61212025-08-26T21:51:02.123ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "67e695b8-615d-4c28-a45e-f0743e3ca581", "content-length": "691", "date": "Tue, 26 Aug 2025 21:51:02 GMT"} })
61222025-08-26T21:51:02.123ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
61232025-08-26T21:51:02.123ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
61242025-08-26T21:51:02.123ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
61252025-08-26T21:51:02.124ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 449.466972ms iteration = 11 status = {}
61262025-08-26T21:51:02.140ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
61272025-08-26T21:51:02.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
61282025-08-26T21:51:02.142ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 476.549613ms iteration = 11 status = null
61292025-08-26T21:51:02.198ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
61302025-08-26T21:51:02.198ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "ae887416-1238-4b16-9476-d1855f4c4a0c", "date": "Tue, 26 Aug 2025 21:51:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
61312025-08-26T21:51:02.198ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
61322025-08-26T21:51:02.200ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "198f898d-c6a3-4160-9052-5c57d02fa8a4", "date": "Tue, 26 Aug 2025 21:51:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
61332025-08-26T21:51:02.200ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
61342025-08-26T21:51:02.201ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
61352025-08-26T21:51:02.202ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 536.208837ms iteration = 8 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
61362025-08-26T21:51:02.242ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
61372025-08-26T21:51:02.318ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
61382025-08-26T21:51:02.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
61392025-08-26T21:51:02.320ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
61402025-08-26T21:51:02.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
61412025-08-26T21:51:02.322ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61422025-08-26T21:51:02.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
61432025-08-26T21:51:02.327ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
61442025-08-26T21:51:02.328ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 662.414281ms iteration = 11 status = {}
61452025-08-26T21:51:02.328ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "4682352f-12e9-4b64-9e67-c9ced5e2addd", "date": "Tue, 26 Aug 2025 21:51:02 GMT"} })
61462025-08-26T21:51:02.333ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:491
61472025-08-26T21:51:02.333ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 646.674626ms
61482025-08-26T21:51:02.333ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 646.690977ms iteration = 2 status = {}
61492025-08-26T21:51:02.346ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
61502025-08-26T21:51:02.346ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61512025-08-26T21:51:02.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61522025-08-26T21:51:02.347ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "45fa3a5d-67f9-45b5-afd0-ef5cd4e89d74", "date": "Tue, 26 Aug 2025 21:51:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
61532025-08-26T21:51:02.347ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
61542025-08-26T21:51:02.348ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "277158b5-f90c-4aa4-8bde-581d0c0f25c1", "date": "Tue, 26 Aug 2025 21:51:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
61552025-08-26T21:51:02.348ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
61562025-08-26T21:51:02.349ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
61572025-08-26T21:51:02.349ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61582025-08-26T21:51:02.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61592025-08-26T21:51:02.350ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "4ccb0976-42d5-409f-8a41-e7e2555db2fc", "date": "Tue, 26 Aug 2025 21:51:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
61602025-08-26T21:51:02.350ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
61612025-08-26T21:51:02.351ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
61622025-08-26T21:51:02.351ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 686.133916ms iteration = 8 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
61632025-08-26T21:51:02.360ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
61642025-08-26T21:51:02.360ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
61652025-08-26T21:51:02.360ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
61662025-08-26T21:51:02.360ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
61672025-08-26T21:51:02.360ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
61682025-08-26T21:51:02.360ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
61692025-08-26T21:51:02.360ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
61702025-08-26T21:51:02.360ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
61712025-08-26T21:51:02.360ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
61722025-08-26T21:51:02.360ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
61732025-08-26T21:51:02.360ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
61742025-08-26T21:51:02.360ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61752025-08-26T21:51:02.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61762025-08-26T21:51:02.363ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
61772025-08-26T21:51:02.363ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
61782025-08-26T21:51:02.516ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "f1ae7b2f-cf5e-45d7-be38-7982ce759948", "date": "Tue, 26 Aug 2025 21:51:02 GMT"} })
61792025-08-26T21:51:02.516ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
61802025-08-26T21:51:02.516ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
61812025-08-26T21:51:02.516ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "4aa70185-e872-4aaa-9ef4-1ab0fdc764e7", "content-length": "682", "date": "Tue, 26 Aug 2025 21:51:02 GMT"} })
61822025-08-26T21:51:02.516ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61832025-08-26T21:51:02.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61842025-08-26T21:51:02.520ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61852025-08-26T21:51:02.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61862025-08-26T21:51:02.524ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61872025-08-26T21:51:02.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61882025-08-26T21:51:02.525ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
61892025-08-26T21:51:02.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
61902025-08-26T21:51:02.533ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 857.553151ms iteration = 11 status = {}
61912025-08-26T21:51:05.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 404 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = aaf4d242-fb19-45cb-b5e7-254f2d312306 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
61922025-08-26T21:51:06.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 62 reason = Timeout
61932025-08-26T21:51:06.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
61942025-08-26T21:51:06.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
61952025-08-26T21:51:06.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61962025-08-26T21:51:06.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
61972025-08-26T21:51:06.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
61982025-08-26T21:51:06.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
61992025-08-26T21:51:06.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
62002025-08-26T21:51:06.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.999088ms iteration = 62 status = {"chicken_switches_updated":false}
62012025-08-26T21:51:11.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 63 reason = Timeout
62022025-08-26T21:51:11.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
62032025-08-26T21:51:11.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
62042025-08-26T21:51:11.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62052025-08-26T21:51:11.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
62062025-08-26T21:51:11.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
62072025-08-26T21:51:11.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
62082025-08-26T21:51:11.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
62092025-08-26T21:51:11.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.525ms iteration = 63 status = {"chicken_switches_updated":false}
62102025-08-26T21:51:11.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 32 reason = Timeout
62112025-08-26T21:51:11.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62122025-08-26T21:51:11.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62132025-08-26T21:51:11.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62142025-08-26T21:51:11.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
62152025-08-26T21:51:11.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62162025-08-26T21:51:11.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62172025-08-26T21:51:11.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62182025-08-26T21:51:11.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62192025-08-26T21:51:11.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62202025-08-26T21:51:11.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
62212025-08-26T21:51:11.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62222025-08-26T21:51:11.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62232025-08-26T21:51:11.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 32 reason = Timeout
62242025-08-26T21:51:11.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
62252025-08-26T21:51:11.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
62262025-08-26T21:51:11.698ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
62272025-08-26T21:51:11.698ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.297375ms iteration = 32 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
62282025-08-26T21:51:11.729ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
62292025-08-26T21:51:11.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
62302025-08-26T21:51:11.766ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 79.39566ms iteration = 32 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
62312025-08-26T21:51:15.346ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 7c667ef3-2377-468f-b14e-d008b66714ad roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
62322025-08-26T21:51:15.346ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 7c667ef3-2377-468f-b14e-d008b66714ad resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
62332025-08-26T21:51:15.350ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 4122 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 7c667ef3-2377-468f-b14e-d008b66714ad response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
62342025-08-26T21:51:15.350ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 7c520932-f377-4056-9048-8de2d5e2698d roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
62352025-08-26T21:51:15.351ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 7c520932-f377-4056-9048-8de2d5e2698d resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
62362025-08-26T21:51:15.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 380 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 52ed5465-df1c-4e20-ae6f-caa6d54e197b response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
62372025-08-26T21:51:15.353ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3069 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 7c520932-f377-4056-9048-8de2d5e2698d response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
62382025-08-26T21:51:16.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 64 reason = Timeout
62392025-08-26T21:51:16.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
62402025-08-26T21:51:16.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
62412025-08-26T21:51:16.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62422025-08-26T21:51:16.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
62432025-08-26T21:51:16.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
62442025-08-26T21:51:16.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
62452025-08-26T21:51:16.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
62462025-08-26T21:51:16.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.472366ms iteration = 64 status = {"chicken_switches_updated":false}
62472025-08-26T21:51:21.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 65 reason = Timeout
62482025-08-26T21:51:21.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
62492025-08-26T21:51:21.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
62502025-08-26T21:51:21.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62512025-08-26T21:51:21.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
62522025-08-26T21:51:21.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
62532025-08-26T21:51:21.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
62542025-08-26T21:51:21.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
62552025-08-26T21:51:21.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.387646ms iteration = 65 status = {"chicken_switches_updated":false}
62562025-08-26T21:51:21.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 33 reason = Timeout
62572025-08-26T21:51:21.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62582025-08-26T21:51:21.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62592025-08-26T21:51:21.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62602025-08-26T21:51:21.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
62612025-08-26T21:51:21.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62622025-08-26T21:51:21.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62632025-08-26T21:51:21.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62642025-08-26T21:51:21.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62652025-08-26T21:51:21.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62662025-08-26T21:51:21.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
62672025-08-26T21:51:21.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62682025-08-26T21:51:21.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
62692025-08-26T21:51:21.687ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 33 reason = Timeout
62702025-08-26T21:51:21.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
62712025-08-26T21:51:21.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
62722025-08-26T21:51:21.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
62732025-08-26T21:51:21.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
62742025-08-26T21:51:21.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.626512ms iteration = 33 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
62752025-08-26T21:51:21.704ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
62762025-08-26T21:51:21.704ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.143047ms iteration = 33 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
62772025-08-26T21:51:25.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 367 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 2e746a59-432f-4757-b7c8-f60c30cf8e06 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
62782025-08-26T21:51:26.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 66 reason = Timeout
62792025-08-26T21:51:26.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
62802025-08-26T21:51:26.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
62812025-08-26T21:51:26.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62822025-08-26T21:51:26.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
62832025-08-26T21:51:26.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
62842025-08-26T21:51:26.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
62852025-08-26T21:51:26.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
62862025-08-26T21:51:26.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.531577ms iteration = 66 status = {"chicken_switches_updated":false}
62872025-08-26T21:51:31.665ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 12 reason = Timeout
62882025-08-26T21:51:31.665ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 12 reason = Timeout
62892025-08-26T21:51:31.665ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 12 reason = Timeout
62902025-08-26T21:51:31.665ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
62912025-08-26T21:51:31.665ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
62922025-08-26T21:51:31.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
62932025-08-26T21:51:31.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
62942025-08-26T21:51:31.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 67 reason = Timeout
62952025-08-26T21:51:31.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
62962025-08-26T21:51:31.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
62972025-08-26T21:51:31.667ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1195, tv_nsec: 525907338 } })
62982025-08-26T21:51:31.668ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
62992025-08-26T21:51:31.668ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1195, tv_nsec: 526321718 } })
63002025-08-26T21:51:31.668ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
63012025-08-26T21:51:31.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63022025-08-26T21:51:31.669ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.301067ms iteration = 12 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
63032025-08-26T21:51:31.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
63042025-08-26T21:51:31.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
63052025-08-26T21:51:31.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
63062025-08-26T21:51:31.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
63072025-08-26T21:51:31.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.24502ms iteration = 67 status = {"chicken_switches_updated":false}
63082025-08-26T21:51:31.674ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 12 reason = Timeout
63092025-08-26T21:51:31.674ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
63102025-08-26T21:51:31.674ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 13 reason = Timeout
63112025-08-26T21:51:31.674ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
63122025-08-26T21:51:31.674ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 12 reason = Timeout
63132025-08-26T21:51:31.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 34 reason = Timeout
63142025-08-26T21:51:31.674ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 12 reason = Timeout
63152025-08-26T21:51:31.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
63162025-08-26T21:51:31.674ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
63172025-08-26T21:51:31.674ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
63182025-08-26T21:51:31.674ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 12 reason = Timeout
63192025-08-26T21:51:31.674ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
63202025-08-26T21:51:31.674ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 12 reason = Timeout
63212025-08-26T21:51:31.674ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
63222025-08-26T21:51:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
63232025-08-26T21:51:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
63242025-08-26T21:51:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
63252025-08-26T21:51:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
63262025-08-26T21:51:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
63272025-08-26T21:51:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
63282025-08-26T21:51:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
63292025-08-26T21:51:31.676ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63302025-08-26T21:51:31.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63312025-08-26T21:51:31.676ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63322025-08-26T21:51:31.676ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63332025-08-26T21:51:31.676ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63342025-08-26T21:51:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
63352025-08-26T21:51:31.680ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
63362025-08-26T21:51:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
63372025-08-26T21:51:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
63382025-08-26T21:51:31.680ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
63392025-08-26T21:51:31.680ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
63402025-08-26T21:51:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
63412025-08-26T21:51:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
63422025-08-26T21:51:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
63432025-08-26T21:51:31.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
63442025-08-26T21:51:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
63452025-08-26T21:51:31.681ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
63462025-08-26T21:51:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
63472025-08-26T21:51:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
63482025-08-26T21:51:31.681ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
63492025-08-26T21:51:31.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
63502025-08-26T21:51:31.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
63512025-08-26T21:51:31.682ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
63522025-08-26T21:51:31.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
63532025-08-26T21:51:31.682ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 7.940359ms iteration = 12 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
63542025-08-26T21:51:31.682ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1195, tv_nsec: 540960678 } })
63552025-08-26T21:51:31.683ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
63562025-08-26T21:51:31.683ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
63572025-08-26T21:51:31.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
63582025-08-26T21:51:31.683ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63592025-08-26T21:51:31.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
63602025-08-26T21:51:31.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
63612025-08-26T21:51:31.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63622025-08-26T21:51:31.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
63632025-08-26T21:51:31.685ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
63642025-08-26T21:51:31.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
63652025-08-26T21:51:31.686ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
63662025-08-26T21:51:31.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
63672025-08-26T21:51:31.686ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 12 reason = Timeout
63682025-08-26T21:51:31.686ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 12 reason = Timeout
63692025-08-26T21:51:31.686ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
63702025-08-26T21:51:31.686ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
63712025-08-26T21:51:31.686ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
63722025-08-26T21:51:31.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 34 reason = Timeout
63732025-08-26T21:51:31.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
63742025-08-26T21:51:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
63752025-08-26T21:51:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
63762025-08-26T21:51:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
63772025-08-26T21:51:31.688ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 12 reason = Timeout
63782025-08-26T21:51:31.688ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 12.181\u{b5}s iteration = 12 status = {"disabled":true,"errors":[],"sps":[]}
63792025-08-26T21:51:31.688ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 12 reason = Timeout
63802025-08-26T21:51:31.688ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 12 reason = Timeout
63812025-08-26T21:51:31.688ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
63822025-08-26T21:51:31.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
63832025-08-26T21:51:31.688ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 12 reason = Timeout
63842025-08-26T21:51:31.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
63852025-08-26T21:51:31.688ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 12 reason = Timeout
63862025-08-26T21:51:31.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
63872025-08-26T21:51:31.688ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 12 reason = Timeout
63882025-08-26T21:51:31.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
63892025-08-26T21:51:31.688ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 12 reason = Timeout
63902025-08-26T21:51:31.688ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
63912025-08-26T21:51:31.688ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
63922025-08-26T21:51:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
63932025-08-26T21:51:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
63942025-08-26T21:51:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
63952025-08-26T21:51:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
63962025-08-26T21:51:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
63972025-08-26T21:51:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
63982025-08-26T21:51:31.689ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63992025-08-26T21:51:31.689ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
64002025-08-26T21:51:31.689ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
64012025-08-26T21:51:31.689ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
64022025-08-26T21:51:31.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
64032025-08-26T21:51:31.690ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
64042025-08-26T21:51:31.690ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
64052025-08-26T21:51:31.690ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
64062025-08-26T21:51:31.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
64072025-08-26T21:51:31.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 2.392294ms iteration = 12 status = {"errors":[],"finish_invoked_ok":[]}
64082025-08-26T21:51:31.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 2.491556ms iteration = 12 status = {"errors":[],"garbage_collect_requested":[]}
64092025-08-26T21:51:31.691ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64102025-08-26T21:51:31.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
64112025-08-26T21:51:31.691ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
64122025-08-26T21:51:31.691ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
64132025-08-26T21:51:31.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
64142025-08-26T21:51:31.695ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
64152025-08-26T21:51:31.695ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
64162025-08-26T21:51:31.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
64172025-08-26T21:51:31.701ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
64182025-08-26T21:51:31.701ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
64192025-08-26T21:51:31.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
64202025-08-26T21:51:31.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
64212025-08-26T21:51:31.703ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 15.425718ms iteration = 12 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
64222025-08-26T21:51:31.704ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64232025-08-26T21:51:31.706ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 18.055807ms iteration = 12 status = {"errors":[],"requests_created_ok":[]}
64242025-08-26T21:51:31.706ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64252025-08-26T21:51:31.708ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
64262025-08-26T21:51:31.708ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
64272025-08-26T21:51:31.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
64282025-08-26T21:51:31.710ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
64292025-08-26T21:51:31.710ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 23.189123ms iteration = 12 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
64302025-08-26T21:51:31.710ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64312025-08-26T21:51:31.710ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
64322025-08-26T21:51:31.710ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
64332025-08-26T21:51:31.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
64342025-08-26T21:51:31.715ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
64352025-08-26T21:51:31.715ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
64362025-08-26T21:51:31.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
64372025-08-26T21:51:31.731ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
64382025-08-26T21:51:31.731ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 56.74081ms iteration = 34 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
64392025-08-26T21:51:31.751ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
64402025-08-26T21:51:31.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
64412025-08-26T21:51:31.752ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
64422025-08-26T21:51:31.752ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
64432025-08-26T21:51:31.752ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
64442025-08-26T21:51:31.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
64452025-08-26T21:51:31.754ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64462025-08-26T21:51:31.754ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
64472025-08-26T21:51:31.754ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
64482025-08-26T21:51:31.754ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 67.84481ms iteration = 12 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
64492025-08-26T21:51:31.756ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
64502025-08-26T21:51:31.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
64512025-08-26T21:51:31.757ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
64522025-08-26T21:51:31.757ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
64532025-08-26T21:51:31.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
64542025-08-26T21:51:31.758ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 71.408119ms iteration = 34 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
64552025-08-26T21:51:31.763ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
64562025-08-26T21:51:31.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
64572025-08-26T21:51:31.766ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
64582025-08-26T21:51:31.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
64592025-08-26T21:51:31.768ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64602025-08-26T21:51:31.769ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
64612025-08-26T21:51:31.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
64622025-08-26T21:51:31.771ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 83.169574ms iteration = 12 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
64632025-08-26T21:51:31.771ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
64642025-08-26T21:51:31.771ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
64652025-08-26T21:51:31.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
64662025-08-26T21:51:31.773ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 99.118524ms iteration = 12 status = 0
64672025-08-26T21:51:31.783ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
64682025-08-26T21:51:31.783ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
64692025-08-26T21:51:31.783ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2628e257-fa5b-48ba-a134-fca47ccddc56", "content-length": "26", "date": "Tue, 26 Aug 2025 21:51:31 GMT"} })
64702025-08-26T21:51:31.783ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
64712025-08-26T21:51:31.784ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
64722025-08-26T21:51:31.784ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
64732025-08-26T21:51:31.785ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
64742025-08-26T21:51:31.785ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a57aead6-4d1d-4256-8796-6c9a78ec9803", "content-length": "26", "date": "Tue, 26 Aug 2025 21:51:31 GMT"} })
64752025-08-26T21:51:31.785ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
64762025-08-26T21:51:31.787ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
64772025-08-26T21:51:31.794ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
64782025-08-26T21:51:31.794ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
64792025-08-26T21:51:31.795ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "55810dc0-aa16-4286-83c2-ca07b5cadc92", "content-length": "26", "date": "Tue, 26 Aug 2025 21:51:31 GMT"} })
64802025-08-26T21:51:31.795ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
64812025-08-26T21:51:31.796ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
64822025-08-26T21:51:31.798ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
64832025-08-26T21:51:31.798ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 123.517364ms iteration = 12 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
64842025-08-26T21:51:31.844ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 169.866049ms iteration = 13 status = {"physical_disks_added":0}
64852025-08-26T21:51:31.845ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
64862025-08-26T21:51:31.845ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
64872025-08-26T21:51:31.845ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7819a64-1d85-4521-ab5a-44ade94f08ed", "content-length": "372", "date": "Tue, 26 Aug 2025 21:51:31 GMT"} })
64882025-08-26T21:51:31.845ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
64892025-08-26T21:51:31.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
64902025-08-26T21:51:31.855ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
64912025-08-26T21:51:31.856ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3215699-e610-4f33-a3b7-52cadd23bf26", "content-length": "691", "date": "Tue, 26 Aug 2025 21:51:31 GMT"} })
64922025-08-26T21:51:31.856ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
64932025-08-26T21:51:31.856ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
64942025-08-26T21:51:31.856ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
64952025-08-26T21:51:31.857ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 183.18973ms iteration = 12 status = {}
64962025-08-26T21:51:31.861ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
64972025-08-26T21:51:31.861ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
64982025-08-26T21:51:31.861ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "a36f8b03-65d2-492d-a968-328fcb61e563", "date": "Tue, 26 Aug 2025 21:51:31 GMT"} })
64992025-08-26T21:51:31.861ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
65002025-08-26T21:51:31.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
65012025-08-26T21:51:31.864ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
65022025-08-26T21:51:31.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
65032025-08-26T21:51:31.868ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
65042025-08-26T21:51:31.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
65052025-08-26T21:51:31.871ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
65062025-08-26T21:51:31.871ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
65072025-08-26T21:51:31.872ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "e867af0c-c948-4027-9ab9-21b18dedfa25", "date": "Tue, 26 Aug 2025 21:51:31 GMT"} })
65082025-08-26T21:51:31.874ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 185.579713ms iteration = 12 status = {}
65092025-08-26T21:51:31.893ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 228.028791ms iteration = 12 status = {}
65102025-08-26T21:51:31.902ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
65112025-08-26T21:51:31.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
65122025-08-26T21:51:31.904ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 239.405687ms iteration = 12 status = null
65132025-08-26T21:51:31.958ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
65142025-08-26T21:51:31.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
65152025-08-26T21:51:31.959ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
65162025-08-26T21:51:31.959ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
65172025-08-26T21:51:31.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
65182025-08-26T21:51:31.973ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
65192025-08-26T21:51:31.973ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
65202025-08-26T21:51:31.973ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
65212025-08-26T21:51:31.973ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
65222025-08-26T21:51:31.973ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
65232025-08-26T21:51:31.973ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
65242025-08-26T21:51:31.973ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
65252025-08-26T21:51:31.973ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
65262025-08-26T21:51:31.973ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
65272025-08-26T21:51:31.973ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
65282025-08-26T21:51:31.973ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
65292025-08-26T21:51:31.973ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
65302025-08-26T21:51:31.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
65312025-08-26T21:51:31.976ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
65322025-08-26T21:51:31.976ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
65332025-08-26T21:51:32.090ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "6ff6e17e-d451-4087-bb26-021ed9330c39", "date": "Tue, 26 Aug 2025 21:51:31 GMT"} })
65342025-08-26T21:51:32.090ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
65352025-08-26T21:51:32.090ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
65362025-08-26T21:51:32.091ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "df8d100b-52d9-47a2-a644-c793ae728a0a", "content-length": "682", "date": "Tue, 26 Aug 2025 21:51:31 GMT"} })
65372025-08-26T21:51:32.091ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
65382025-08-26T21:51:32.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
65392025-08-26T21:51:32.095ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
65402025-08-26T21:51:32.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
65412025-08-26T21:51:32.098ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
65422025-08-26T21:51:32.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
65432025-08-26T21:51:32.100ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
65442025-08-26T21:51:32.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
65452025-08-26T21:51:32.107ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 432.818027ms iteration = 12 status = {}
65462025-08-26T21:51:35.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 385 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 04829f32-459a-419e-8240-2dd5c49fce10 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
65472025-08-26T21:51:36.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 68 reason = Timeout
65482025-08-26T21:51:36.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
65492025-08-26T21:51:36.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
65502025-08-26T21:51:36.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65512025-08-26T21:51:36.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
65522025-08-26T21:51:36.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
65532025-08-26T21:51:36.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
65542025-08-26T21:51:36.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
65552025-08-26T21:51:36.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.54329ms iteration = 68 status = {"chicken_switches_updated":false}
65562025-08-26T21:51:41.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 69 reason = Timeout
65572025-08-26T21:51:41.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
65582025-08-26T21:51:41.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
65592025-08-26T21:51:41.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65602025-08-26T21:51:41.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
65612025-08-26T21:51:41.673ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
65622025-08-26T21:51:41.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
65632025-08-26T21:51:41.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 35 reason = Timeout
65642025-08-26T21:51:41.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65652025-08-26T21:51:41.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65662025-08-26T21:51:41.675ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
65672025-08-26T21:51:41.675ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.205345ms iteration = 69 status = {"chicken_switches_updated":false}
65682025-08-26T21:51:41.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65692025-08-26T21:51:41.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
65702025-08-26T21:51:41.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65712025-08-26T21:51:41.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65722025-08-26T21:51:41.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65732025-08-26T21:51:41.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65742025-08-26T21:51:41.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65752025-08-26T21:51:41.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
65762025-08-26T21:51:41.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65772025-08-26T21:51:41.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
65782025-08-26T21:51:41.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 35 reason = Timeout
65792025-08-26T21:51:41.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
65802025-08-26T21:51:41.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
65812025-08-26T21:51:41.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
65822025-08-26T21:51:41.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
65832025-08-26T21:51:41.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.419247ms iteration = 35 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
65842025-08-26T21:51:41.703ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
65852025-08-26T21:51:41.703ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.287168ms iteration = 35 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
65862025-08-26T21:51:45.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 45f21499-d6e8-4a6b-a9d8-528eee7de4b6 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
65872025-08-26T21:51:45.346ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 45f21499-d6e8-4a6b-a9d8-528eee7de4b6 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
65882025-08-26T21:51:45.348ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2852 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 45f21499-d6e8-4a6b-a9d8-528eee7de4b6 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
65892025-08-26T21:51:45.348ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 14b7d904-d45c-4124-b180-914ab87bfe45 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
65902025-08-26T21:51:45.349ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 14b7d904-d45c-4124-b180-914ab87bfe45 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
65912025-08-26T21:51:45.350ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 1898 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 14b7d904-d45c-4124-b180-914ab87bfe45 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
65922025-08-26T21:51:45.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 287 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 0def2d9c-4886-47f3-968c-815e14acf430 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
65932025-08-26T21:51:46.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 70 reason = Timeout
65942025-08-26T21:51:46.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
65952025-08-26T21:51:46.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
65962025-08-26T21:51:46.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65972025-08-26T21:51:46.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
65982025-08-26T21:51:46.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
65992025-08-26T21:51:46.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
66002025-08-26T21:51:46.675ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
66012025-08-26T21:51:46.675ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.604851ms iteration = 70 status = {"chicken_switches_updated":false}
66022025-08-26T21:51:51.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 71 reason = Timeout
66032025-08-26T21:51:51.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
66042025-08-26T21:51:51.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
66052025-08-26T21:51:51.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66062025-08-26T21:51:51.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
66072025-08-26T21:51:51.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
66082025-08-26T21:51:51.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
66092025-08-26T21:51:51.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
66102025-08-26T21:51:51.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.243633ms iteration = 71 status = {"chicken_switches_updated":false}
66112025-08-26T21:51:51.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
66122025-08-26T21:51:51.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66132025-08-26T21:51:51.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66142025-08-26T21:51:51.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66152025-08-26T21:51:51.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
66162025-08-26T21:51:51.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66172025-08-26T21:51:51.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66182025-08-26T21:51:51.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66192025-08-26T21:51:51.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66202025-08-26T21:51:51.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66212025-08-26T21:51:51.687ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 36 reason = Timeout
66222025-08-26T21:51:51.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
66232025-08-26T21:51:51.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
66242025-08-26T21:51:51.687ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
66252025-08-26T21:51:51.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
66262025-08-26T21:51:51.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
66272025-08-26T21:51:51.689ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
66282025-08-26T21:51:51.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
66292025-08-26T21:51:51.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.582511ms iteration = 36 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
66302025-08-26T21:51:51.701ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
66312025-08-26T21:51:51.701ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.596707ms iteration = 36 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
66322025-08-26T21:51:55.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 408 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = a4727dda-d6b7-4e2b-88c8-052b752dbec7 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
66332025-08-26T21:51:56.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 72 reason = Timeout
66342025-08-26T21:51:56.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
66352025-08-26T21:51:56.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
66362025-08-26T21:51:56.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66372025-08-26T21:51:56.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
66382025-08-26T21:51:56.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
66392025-08-26T21:51:56.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
66402025-08-26T21:51:56.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
66412025-08-26T21:51:56.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.764133ms iteration = 72 status = {"chicken_switches_updated":false}
66422025-08-26T21:52:00.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 866d4f48-e8e1-4301-b485-d344de775b79 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
66432025-08-26T21:52:00.345ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 866d4f48-e8e1-4301-b485-d344de775b79 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
66442025-08-26T21:52:00.348ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3250 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 866d4f48-e8e1-4301-b485-d344de775b79 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
66452025-08-26T21:52:00.349ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = ac1e6663-6a24-4953-bbe4-5cf61fc1d4e4 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
66462025-08-26T21:52:00.349ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = ac1e6663-6a24-4953-bbe4-5cf61fc1d4e4 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
66472025-08-26T21:52:00.351ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2217 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = ac1e6663-6a24-4953-bbe4-5cf61fc1d4e4 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
66482025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
66492025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
66502025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
66512025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
66522025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
66532025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
66542025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 13 reason = Timeout
66552025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
66562025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
66572025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
66582025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 7 reason = Timeout
66592025-08-26T21:52:01.665ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:42:01.665468978 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
66602025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
66612025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
66622025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
66632025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
66642025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
66652025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
66662025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
66672025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
66682025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 8 reason = Timeout
66692025-08-26T21:52:01.665ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
66702025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 9.8\u{b5}s iteration = 8 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
66712025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
66722025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
66732025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
66742025-08-26T21:52:01.665ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1225, tv_nsec: 523950474 } })
66752025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
66762025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
66772025-08-26T21:52:01.665ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1225, tv_nsec: 524023726 } })
66782025-08-26T21:52:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
66792025-08-26T21:52:01.666ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
66802025-08-26T21:52:01.666ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 978.782\u{b5}s iteration = 7 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
66812025-08-26T21:52:01.666ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
66822025-08-26T21:52:01.666ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
66832025-08-26T21:52:01.666ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.068155ms iteration = 7 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
66842025-08-26T21:52:01.666ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
66852025-08-26T21:52:01.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 73 reason = Timeout
66862025-08-26T21:52:01.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
66872025-08-26T21:52:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
66882025-08-26T21:52:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
66892025-08-26T21:52:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
66902025-08-26T21:52:01.667ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66912025-08-26T21:52:01.667ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:42:01.665468978 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
66922025-08-26T21:52:01.667ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1225, tv_nsec: 525899488 } })
66932025-08-26T21:52:01.667ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 2.383214ms iteration = 7 status = {"expiration":"2025-08-26T21:42:01.665468978Z","pruned":[]}
66942025-08-26T21:52:01.667ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1225, tv_nsec: 526011591 } })
66952025-08-26T21:52:01.668ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
66962025-08-26T21:52:01.668ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.793093ms iteration = 13 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
66972025-08-26T21:52:01.668ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
66982025-08-26T21:52:01.668ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66992025-08-26T21:52:01.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67002025-08-26T21:52:01.668ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67012025-08-26T21:52:01.669ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
67022025-08-26T21:52:01.670ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
67032025-08-26T21:52:01.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
67042025-08-26T21:52:01.671ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
67052025-08-26T21:52:01.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
67062025-08-26T21:52:01.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
67072025-08-26T21:52:01.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
67082025-08-26T21:52:01.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
67092025-08-26T21:52:01.673ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67102025-08-26T21:52:01.673ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 9 reason = Timeout
67112025-08-26T21:52:01.673ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
67122025-08-26T21:52:01.673ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 11.63\u{b5}s iteration = 9 status = "Disabled"
67132025-08-26T21:52:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
67142025-08-26T21:52:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
67152025-08-26T21:52:01.674ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 4 reason = Timeout
67162025-08-26T21:52:01.674ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 13 reason = Timeout
67172025-08-26T21:52:01.674ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
67182025-08-26T21:52:01.674ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 13 reason = Timeout
67192025-08-26T21:52:01.674ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
67202025-08-26T21:52:01.674ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
67212025-08-26T21:52:01.674ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 13 reason = Timeout
67222025-08-26T21:52:01.674ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
67232025-08-26T21:52:01.674ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 13 reason = Timeout
67242025-08-26T21:52:01.674ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
67252025-08-26T21:52:01.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
67262025-08-26T21:52:01.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67272025-08-26T21:52:01.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
67282025-08-26T21:52:01.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.080502ms iteration = 73 status = {"chicken_switches_updated":false}
67292025-08-26T21:52:01.674ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 14 reason = Timeout
67302025-08-26T21:52:01.674ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
67312025-08-26T21:52:01.674ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 7 reason = Timeout
67322025-08-26T21:52:01.674ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
67332025-08-26T21:52:01.674ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
67342025-08-26T21:52:01.674ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
67352025-08-26T21:52:01.674ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
67362025-08-26T21:52:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
67372025-08-26T21:52:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
67382025-08-26T21:52:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
67392025-08-26T21:52:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
67402025-08-26T21:52:01.675ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
67412025-08-26T21:52:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
67422025-08-26T21:52:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
67432025-08-26T21:52:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
67442025-08-26T21:52:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67452025-08-26T21:52:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
67462025-08-26T21:52:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
67472025-08-26T21:52:01.675ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
67482025-08-26T21:52:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
67492025-08-26T21:52:01.675ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
67502025-08-26T21:52:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
67512025-08-26T21:52:01.675ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
67522025-08-26T21:52:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
67532025-08-26T21:52:01.677ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
67542025-08-26T21:52:01.677ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
67552025-08-26T21:52:01.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
67562025-08-26T21:52:01.677ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67572025-08-26T21:52:01.677ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67582025-08-26T21:52:01.678ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67592025-08-26T21:52:01.678ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67602025-08-26T21:52:01.678ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67612025-08-26T21:52:01.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67622025-08-26T21:52:01.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
67632025-08-26T21:52:01.681ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
67642025-08-26T21:52:01.681ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
67652025-08-26T21:52:01.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
67662025-08-26T21:52:01.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
67672025-08-26T21:52:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
67682025-08-26T21:52:01.682ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
67692025-08-26T21:52:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
67702025-08-26T21:52:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
67712025-08-26T21:52:01.683ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
67722025-08-26T21:52:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
67732025-08-26T21:52:01.683ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
67742025-08-26T21:52:01.683ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 9.051934ms iteration = 13 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
67752025-08-26T21:52:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
67762025-08-26T21:52:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
67772025-08-26T21:52:01.684ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67782025-08-26T21:52:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
67792025-08-26T21:52:01.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67802025-08-26T21:52:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67812025-08-26T21:52:01.684ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1225, tv_nsec: 542651696 } })
67822025-08-26T21:52:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
67832025-08-26T21:52:01.684ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
67842025-08-26T21:52:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
67852025-08-26T21:52:01.685ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
67862025-08-26T21:52:01.685ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
67872025-08-26T21:52:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
67882025-08-26T21:52:01.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67892025-08-26T21:52:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67902025-08-26T21:52:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
67912025-08-26T21:52:01.686ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
67922025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
67932025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
67942025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67952025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 7 reason = Timeout
67962025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
67972025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 13 reason = Timeout
67982025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 37 reason = Timeout
67992025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 13 reason = Timeout
68002025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
68012025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
68022025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
68032025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
68042025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
68052025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
68062025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
68072025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 7 reason = Timeout
68082025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 13 reason = Timeout
68092025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
68102025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
68112025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
68122025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 13 reason = Timeout
68132025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
68142025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 13 reason = Timeout
68152025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
68162025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
68172025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 13 reason = Timeout
68182025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 13 reason = Timeout
68192025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
68202025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
68212025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 13 reason = Timeout
68222025-08-26T21:52:01.687ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
68232025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
68242025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 7 reason = Timeout
68252025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
68262025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 7 reason = Timeout
68272025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 7 reason = Timeout
68282025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
68292025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
68302025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 13 reason = Timeout
68312025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 19.351\u{b5}s iteration = 13 status = {"disabled":true,"errors":[],"sps":[]}
68322025-08-26T21:52:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
68332025-08-26T21:52:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
68342025-08-26T21:52:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
68352025-08-26T21:52:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
68362025-08-26T21:52:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
68372025-08-26T21:52:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
68382025-08-26T21:52:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
68392025-08-26T21:52:01.688ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
68402025-08-26T21:52:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
68412025-08-26T21:52:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
68422025-08-26T21:52:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
68432025-08-26T21:52:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
68442025-08-26T21:52:01.688ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68452025-08-26T21:52:01.689ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68462025-08-26T21:52:01.689ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
68472025-08-26T21:52:01.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
68482025-08-26T21:52:01.691ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 3.264783ms iteration = 13 status = {"errors":[],"garbage_collect_requested":[]}
68492025-08-26T21:52:01.691ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
68502025-08-26T21:52:01.691ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
68512025-08-26T21:52:01.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
68522025-08-26T21:52:01.693ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
68532025-08-26T21:52:01.693ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
68542025-08-26T21:52:01.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
68552025-08-26T21:52:01.693ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
68562025-08-26T21:52:01.693ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
68572025-08-26T21:52:01.694ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 6.456356ms iteration = 7 status = {"errors":[],"found_port_ok":[]}
68582025-08-26T21:52:01.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
68592025-08-26T21:52:01.700ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
68602025-08-26T21:52:01.700ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
68612025-08-26T21:52:01.700ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 12.800808ms iteration = 7 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
68622025-08-26T21:52:01.703ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68632025-08-26T21:52:01.705ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 18.374514ms iteration = 13 status = {"errors":[],"requests_created_ok":[]}
68642025-08-26T21:52:01.706ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
68652025-08-26T21:52:01.706ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 18.802794ms iteration = 7 status = {"by_rx":{},"error":null}
68662025-08-26T21:52:01.707ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
68672025-08-26T21:52:01.708ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
68682025-08-26T21:52:01.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
68692025-08-26T21:52:01.710ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 23.329786ms iteration = 37 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
68702025-08-26T21:52:01.712ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
68712025-08-26T21:52:01.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
68722025-08-26T21:52:01.723ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
68732025-08-26T21:52:01.724ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 36.850181ms iteration = 13 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
68742025-08-26T21:52:01.726ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
68752025-08-26T21:52:01.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
68762025-08-26T21:52:01.727ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 39.425959ms iteration = 13 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
68772025-08-26T21:52:01.728ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 40.559995ms iteration = 7 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
68782025-08-26T21:52:01.728ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68792025-08-26T21:52:01.732ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
68802025-08-26T21:52:01.732ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
68812025-08-26T21:52:01.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
68822025-08-26T21:52:01.752ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68832025-08-26T21:52:01.755ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
68842025-08-26T21:52:01.755ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
68852025-08-26T21:52:01.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
68862025-08-26T21:52:01.759ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
68872025-08-26T21:52:01.759ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
68882025-08-26T21:52:01.759ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 84.510945ms iteration = 37 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
68892025-08-26T21:52:01.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
68902025-08-26T21:52:01.761ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68912025-08-26T21:52:01.764ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 77.093338ms iteration = 13 status = {"errors":[],"finish_invoked_ok":[]}
68922025-08-26T21:52:01.765ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
68932025-08-26T21:52:01.765ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
68942025-08-26T21:52:01.765ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
68952025-08-26T21:52:01.765ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
68962025-08-26T21:52:01.765ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
68972025-08-26T21:52:01.765ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
68982025-08-26T21:52:01.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
68992025-08-26T21:52:01.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
69002025-08-26T21:52:01.766ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
69012025-08-26T21:52:01.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
69022025-08-26T21:52:01.767ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69032025-08-26T21:52:01.768ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
69042025-08-26T21:52:01.768ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
69052025-08-26T21:52:01.768ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 93.308003ms iteration = 7 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
69062025-08-26T21:52:01.769ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
69072025-08-26T21:52:01.769ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
69082025-08-26T21:52:01.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
69092025-08-26T21:52:01.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
69102025-08-26T21:52:01.770ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
69112025-08-26T21:52:01.770ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
69122025-08-26T21:52:01.770ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
69132025-08-26T21:52:01.770ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
69142025-08-26T21:52:01.770ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
69152025-08-26T21:52:01.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
69162025-08-26T21:52:01.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
69172025-08-26T21:52:01.771ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69182025-08-26T21:52:01.771ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 83.626065ms iteration = 13 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
69192025-08-26T21:52:01.771ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69202025-08-26T21:52:01.772ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
69212025-08-26T21:52:01.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
69222025-08-26T21:52:01.773ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
69232025-08-26T21:52:01.773ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
69242025-08-26T21:52:01.773ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
69252025-08-26T21:52:01.774ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69262025-08-26T21:52:01.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
69272025-08-26T21:52:01.774ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69282025-08-26T21:52:01.775ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
69292025-08-26T21:52:01.775ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
69302025-08-26T21:52:01.775ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
69312025-08-26T21:52:01.775ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
69322025-08-26T21:52:01.775ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
69332025-08-26T21:52:01.775ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
69342025-08-26T21:52:01.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
69352025-08-26T21:52:01.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
69362025-08-26T21:52:01.776ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
69372025-08-26T21:52:01.776ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
69382025-08-26T21:52:01.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
69392025-08-26T21:52:01.777ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69402025-08-26T21:52:01.777ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
69412025-08-26T21:52:01.777ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
69422025-08-26T21:52:01.777ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 90.417118ms iteration = 13 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
69432025-08-26T21:52:01.778ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 103.657497ms iteration = 4 status = {"nsuccess":5}
69442025-08-26T21:52:01.778ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
69452025-08-26T21:52:01.778ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
69462025-08-26T21:52:01.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
69472025-08-26T21:52:01.779ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
69482025-08-26T21:52:01.779ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 0 version = 2
69492025-08-26T21:52:01.779ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
69502025-08-26T21:52:01.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
69512025-08-26T21:52:01.780ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
69522025-08-26T21:52:01.780ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
69532025-08-26T21:52:01.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
69542025-08-26T21:52:01.781ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69552025-08-26T21:52:01.782ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
69562025-08-26T21:52:01.782ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
69572025-08-26T21:52:01.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
69582025-08-26T21:52:01.782ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 107.870581ms iteration = 13 status = 0
69592025-08-26T21:52:01.784ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
69602025-08-26T21:52:01.784ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
69612025-08-26T21:52:01.784ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
69622025-08-26T21:52:01.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
69632025-08-26T21:52:01.785ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
69642025-08-26T21:52:01.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
69652025-08-26T21:52:01.786ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
69662025-08-26T21:52:01.786ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
69672025-08-26T21:52:01.786ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69682025-08-26T21:52:01.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
69692025-08-26T21:52:01.787ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69702025-08-26T21:52:01.790ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
69712025-08-26T21:52:01.791ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
69722025-08-26T21:52:01.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
69732025-08-26T21:52:01.791ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 4 version = 2
69742025-08-26T21:52:01.791ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
69752025-08-26T21:52:01.791ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-08-26 21:46:05.296205 UTC dns_group = external generation = 2
69762025-08-26T21:52:01.791ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 126.215085ms iteration = 7 status = {"generation":2}
69772025-08-26T21:52:01.792ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
69782025-08-26T21:52:01.792ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
69792025-08-26T21:52:01.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
69802025-08-26T21:52:01.796ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69812025-08-26T21:52:01.798ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
69822025-08-26T21:52:01.798ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
69832025-08-26T21:52:01.798ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-08-26 21:46:04.065217 UTC dns_group = internal generation = 1
69842025-08-26T21:52:01.798ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 132.974688ms iteration = 7 status = {"generation":1}
69852025-08-26T21:52:01.799ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
69862025-08-26T21:52:01.799ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
69872025-08-26T21:52:01.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
69882025-08-26T21:52:01.826ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-08-26T21:45:59.866217Z, time_modified: 2025-08-26T21:45:59.866217Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }) }
69892025-08-26T21:52:01.826ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 161.47598ms iteration = 7 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]}},"default_endpoint":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
69902025-08-26T21:52:01.828ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
69912025-08-26T21:52:01.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
69922025-08-26T21:52:01.834ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69932025-08-26T21:52:01.834ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
69942025-08-26T21:52:01.834ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
69952025-08-26T21:52:01.834ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3241c3a-d4d0-4674-9e6e-efb45b74ac24", "content-length": "26", "date": "Tue, 26 Aug 2025 21:52:01 GMT"} })
69962025-08-26T21:52:01.834ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
69972025-08-26T21:52:01.836ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
69982025-08-26T21:52:01.836ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
69992025-08-26T21:52:01.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
70002025-08-26T21:52:01.836ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
70012025-08-26T21:52:01.838ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
70022025-08-26T21:52:01.839ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 164.289354ms iteration = 13 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
70032025-08-26T21:52:01.858ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
70042025-08-26T21:52:01.886ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
70052025-08-26T21:52:01.888ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
70062025-08-26T21:52:01.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
70072025-08-26T21:52:01.893ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
70082025-08-26T21:52:01.894ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 206.882523ms iteration = 7 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
70092025-08-26T21:52:01.917ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 242.509097ms iteration = 14 status = {"physical_disks_added":0}
70102025-08-26T21:52:01.949ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
70112025-08-26T21:52:01.949ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
70122025-08-26T21:52:01.949ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "77dff7cf-79ab-42dc-85b5-8ec9bd58c73c", "content-length": "372", "date": "Tue, 26 Aug 2025 21:52:01 GMT"} })
70132025-08-26T21:52:01.949ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
70142025-08-26T21:52:01.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
70152025-08-26T21:52:01.968ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
70162025-08-26T21:52:01.968ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
70172025-08-26T21:52:01.968ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "a2724355-4215-4bc6-9dd9-5557f85249aa", "date": "Tue, 26 Aug 2025 21:52:01 GMT"} })
70182025-08-26T21:52:01.968ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
70192025-08-26T21:52:01.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
70202025-08-26T21:52:01.970ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
70212025-08-26T21:52:01.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
70222025-08-26T21:52:01.973ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
70232025-08-26T21:52:01.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
70242025-08-26T21:52:01.977ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
70252025-08-26T21:52:01.977ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
70262025-08-26T21:52:01.978ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c7aa0b61-8f61-478d-a608-ce4f9c099e9f", "date": "Tue, 26 Aug 2025 21:52:01 GMT"} })
70272025-08-26T21:52:01.980ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 292.386701ms iteration = 13 status = {}
70282025-08-26T21:52:01.982ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
70292025-08-26T21:52:01.982ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
70302025-08-26T21:52:01.982ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "96cd9dcd-b559-4dcc-a58e-fffecb7d652b", "content-length": "26", "date": "Tue, 26 Aug 2025 21:52:01 GMT"} })
70312025-08-26T21:52:01.982ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
70322025-08-26T21:52:01.983ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
70332025-08-26T21:52:02.018ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
70342025-08-26T21:52:02.018ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
70352025-08-26T21:52:02.018ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a330364b-0cbd-4a7f-a5e6-25cc88d67e1d", "content-length": "26", "date": "Tue, 26 Aug 2025 21:52:01 GMT"} })
70362025-08-26T21:52:02.018ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
70372025-08-26T21:52:02.020ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
70382025-08-26T21:52:02.066ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 400.790905ms iteration = 13 status = {}
70392025-08-26T21:52:02.090ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
70402025-08-26T21:52:02.128ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
70412025-08-26T21:52:02.128ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "16fe693d-88d2-40a0-91d8-d1fb670bd273", "content-length": "691", "date": "Tue, 26 Aug 2025 21:52:01 GMT"} })
70422025-08-26T21:52:02.128ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
70432025-08-26T21:52:02.128ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
70442025-08-26T21:52:02.128ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
70452025-08-26T21:52:02.130ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 455.785755ms iteration = 13 status = {}
70462025-08-26T21:52:02.141ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
70472025-08-26T21:52:02.142ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "4a0f65b2-1ecf-485e-8596-82bd1ea60e37", "date": "Tue, 26 Aug 2025 21:52:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
70482025-08-26T21:52:02.142ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
70492025-08-26T21:52:02.143ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "5f527f0f-123b-4b8a-837e-2921b7c0e691", "date": "Tue, 26 Aug 2025 21:52:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
70502025-08-26T21:52:02.143ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
70512025-08-26T21:52:02.146ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
70522025-08-26T21:52:02.146ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 480.645464ms iteration = 9 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
70532025-08-26T21:52:02.159ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
70542025-08-26T21:52:02.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
70552025-08-26T21:52:02.161ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 496.41186ms iteration = 13 status = null
70562025-08-26T21:52:02.181ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
70572025-08-26T21:52:02.182ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "cc35ca7b-8e99-4399-8a2a-c5a8c1b9902f", "date": "Tue, 26 Aug 2025 21:52:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
70582025-08-26T21:52:02.182ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
70592025-08-26T21:52:02.184ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "d780b62a-dc1d-4baa-ad43-e5b6e765bccf", "date": "Tue, 26 Aug 2025 21:52:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
70602025-08-26T21:52:02.184ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
70612025-08-26T21:52:02.186ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "ec52daad-9117-4d55-9c66-8e9486524466", "date": "Tue, 26 Aug 2025 21:52:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
70622025-08-26T21:52:02.186ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
70632025-08-26T21:52:02.188ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
70642025-08-26T21:52:02.188ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 522.693563ms iteration = 9 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
70652025-08-26T21:52:02.223ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
70662025-08-26T21:52:02.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
70672025-08-26T21:52:02.225ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
70682025-08-26T21:52:02.225ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
70692025-08-26T21:52:02.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
70702025-08-26T21:52:02.233ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
70712025-08-26T21:52:02.233ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
70722025-08-26T21:52:02.233ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
70732025-08-26T21:52:02.233ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
70742025-08-26T21:52:02.233ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
70752025-08-26T21:52:02.233ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
70762025-08-26T21:52:02.233ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
70772025-08-26T21:52:02.233ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
70782025-08-26T21:52:02.233ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
70792025-08-26T21:52:02.233ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
70802025-08-26T21:52:02.234ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
70812025-08-26T21:52:02.234ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
70822025-08-26T21:52:02.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
70832025-08-26T21:52:02.236ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
70842025-08-26T21:52:02.236ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
70852025-08-26T21:52:02.329ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 2
70862025-08-26T21:52:02.329ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221
70872025-08-26T21:52:02.361ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
70882025-08-26T21:52:02.361ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::a]:37515
70892025-08-26T21:52:02.361ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "6c9b435a-5909-445f-906c-05a01a741905", "date": "Tue, 26 Aug 2025 21:52:01 GMT"} })
70902025-08-26T21:52:02.361ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
70912025-08-26T21:52:02.361ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
70922025-08-26T21:52:02.361ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:37515 req_id = f2a7ae1d-d8c2-456b-b23e-656faf5c0144 roles = RoleSet { roles: {} } uri = /metrics/producers
70932025-08-26T21:52:02.361ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:37515 req_id = f2a7ae1d-d8c2-456b-b23e-656faf5c0144 resource = Database result = Ok(()) uri = /metrics/producers
70942025-08-26T21:52:02.362ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9e60794-996f-4878-a8d0-d19cc2942e47", "content-length": "682", "date": "Tue, 26 Aug 2025 21:52:01 GMT"} })
70952025-08-26T21:52:02.362ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
70962025-08-26T21:52:02.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
70972025-08-26T21:52:02.365ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
70982025-08-26T21:52:02.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
70992025-08-26T21:52:02.368ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
71002025-08-26T21:52:02.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
71012025-08-26T21:52:02.370ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
71022025-08-26T21:52:02.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
71032025-08-26T21:52:02.377ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 702.467915ms iteration = 13 status = {}
71042025-08-26T21:52:02.404ZINFOnexus: assigned collector to new producer collector_id = f843ff0e-eef8-49c4-b428-23e15df1def3 file = nexus/src/app/oximeter.rs:76 producer_id = 4996dcc4-c530-4f64-b407-fa372462bd9e
71052025-08-26T21:52:02.404ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 43567 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:37515 req_id = f2a7ae1d-d8c2-456b-b23e-656faf5c0144 response_code = 201 uri = /metrics/producers
71062025-08-26T21:52:02.405ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "f2a7ae1d-d8c2-456b-b23e-656faf5c0144", "content-length": "41", "date": "Tue, 26 Aug 2025 21:52:02 GMT"} })
71072025-08-26T21:52:02.405ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
71082025-08-26T21:52:05.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 413 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 6c778604-383e-4e98-b4fc-5e350d3b06a2 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
71092025-08-26T21:52:06.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 74 reason = Timeout
71102025-08-26T21:52:06.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
71112025-08-26T21:52:06.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
71122025-08-26T21:52:06.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71132025-08-26T21:52:06.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
71142025-08-26T21:52:06.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
71152025-08-26T21:52:06.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
71162025-08-26T21:52:06.675ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
71172025-08-26T21:52:06.675ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.858457ms iteration = 74 status = {"chicken_switches_updated":false}
71182025-08-26T21:52:11.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 75 reason = Timeout
71192025-08-26T21:52:11.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
71202025-08-26T21:52:11.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
71212025-08-26T21:52:11.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71222025-08-26T21:52:11.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
71232025-08-26T21:52:11.673ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
71242025-08-26T21:52:11.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
71252025-08-26T21:52:11.675ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
71262025-08-26T21:52:11.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
71272025-08-26T21:52:11.675ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.901068ms iteration = 75 status = {"chicken_switches_updated":false}
71282025-08-26T21:52:11.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71292025-08-26T21:52:11.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
71302025-08-26T21:52:11.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71312025-08-26T21:52:11.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
71322025-08-26T21:52:11.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71332025-08-26T21:52:11.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
71342025-08-26T21:52:11.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71352025-08-26T21:52:11.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
71362025-08-26T21:52:11.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71372025-08-26T21:52:11.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
71382025-08-26T21:52:11.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71392025-08-26T21:52:11.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
71402025-08-26T21:52:11.687ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 38 reason = Timeout
71412025-08-26T21:52:11.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
71422025-08-26T21:52:11.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
71432025-08-26T21:52:11.689ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
71442025-08-26T21:52:11.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
71452025-08-26T21:52:11.691ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.895928ms iteration = 38 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
71462025-08-26T21:52:11.698ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
71472025-08-26T21:52:11.698ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.222444ms iteration = 38 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
71482025-08-26T21:52:15.346ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = c8ba02d0-4cca-4f96-9134-b3e06b616b42 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
71492025-08-26T21:52:15.346ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = c8ba02d0-4cca-4f96-9134-b3e06b616b42 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
71502025-08-26T21:52:15.349ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3191 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = c8ba02d0-4cca-4f96-9134-b3e06b616b42 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
71512025-08-26T21:52:15.349ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 779238f7-bad2-4e35-9395-ee7b362509e7 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
71522025-08-26T21:52:15.349ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 779238f7-bad2-4e35-9395-ee7b362509e7 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
71532025-08-26T21:52:15.351ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 1807 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 779238f7-bad2-4e35-9395-ee7b362509e7 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
71542025-08-26T21:52:15.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 336 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 6429b93b-b9d4-4626-b94f-1d6eb0c7f926 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
71552025-08-26T21:52:16.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 76 reason = Timeout
71562025-08-26T21:52:16.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
71572025-08-26T21:52:16.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
71582025-08-26T21:52:16.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71592025-08-26T21:52:16.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
71602025-08-26T21:52:16.673ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
71612025-08-26T21:52:16.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
71622025-08-26T21:52:16.675ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
71632025-08-26T21:52:16.675ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.551893ms iteration = 76 status = {"chicken_switches_updated":false}
71642025-08-26T21:52:21.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 77 reason = Timeout
71652025-08-26T21:52:21.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
71662025-08-26T21:52:21.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
71672025-08-26T21:52:21.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71682025-08-26T21:52:21.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
71692025-08-26T21:52:21.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
71702025-08-26T21:52:21.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
71712025-08-26T21:52:21.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
71722025-08-26T21:52:21.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.263394ms iteration = 77 status = {"chicken_switches_updated":false}
71732025-08-26T21:52:21.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
71742025-08-26T21:52:21.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71752025-08-26T21:52:21.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
71762025-08-26T21:52:21.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71772025-08-26T21:52:21.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
71782025-08-26T21:52:21.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71792025-08-26T21:52:21.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
71802025-08-26T21:52:21.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71812025-08-26T21:52:21.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
71822025-08-26T21:52:21.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71832025-08-26T21:52:21.687ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
71842025-08-26T21:52:21.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71852025-08-26T21:52:21.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
71862025-08-26T21:52:21.687ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 39 reason = Timeout
71872025-08-26T21:52:21.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
71882025-08-26T21:52:21.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
71892025-08-26T21:52:21.689ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
71902025-08-26T21:52:21.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
71912025-08-26T21:52:21.691ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.53866ms iteration = 39 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
71922025-08-26T21:52:21.700ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
71932025-08-26T21:52:21.700ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.693807ms iteration = 39 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
71942025-08-26T21:52:25.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 518 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 04deca0c-de12-4634-8788-fb848edb31e8 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
71952025-08-26T21:52:26.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 78 reason = Timeout
71962025-08-26T21:52:26.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
71972025-08-26T21:52:26.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
71982025-08-26T21:52:26.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71992025-08-26T21:52:26.675ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
72002025-08-26T21:52:26.675ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
72012025-08-26T21:52:26.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
72022025-08-26T21:52:26.678ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
72032025-08-26T21:52:26.678ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 11.08932ms iteration = 78 status = {"chicken_switches_updated":false}
72042025-08-26T21:52:31.665ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
72052025-08-26T21:52:31.665ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
72062025-08-26T21:52:31.665ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
72072025-08-26T21:52:31.665ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 14 reason = Timeout
72082025-08-26T21:52:31.665ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
72092025-08-26T21:52:31.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
72102025-08-26T21:52:31.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
72112025-08-26T21:52:31.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 79 reason = Timeout
72122025-08-26T21:52:31.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
72132025-08-26T21:52:31.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
72142025-08-26T21:52:31.667ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1255, tv_nsec: 525993450 } })
72152025-08-26T21:52:31.667ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1255, tv_nsec: 526031761 } })
72162025-08-26T21:52:31.668ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
72172025-08-26T21:52:31.668ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
72182025-08-26T21:52:31.669ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.923549ms iteration = 14 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
72192025-08-26T21:52:31.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72202025-08-26T21:52:31.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
72212025-08-26T21:52:31.673ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
72222025-08-26T21:52:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
72232025-08-26T21:52:31.675ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 14 reason = Timeout
72242025-08-26T21:52:31.675ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 15 reason = Timeout
72252025-08-26T21:52:31.675ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
72262025-08-26T21:52:31.675ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
72272025-08-26T21:52:31.675ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
72282025-08-26T21:52:31.675ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 14 reason = Timeout
72292025-08-26T21:52:31.675ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
72302025-08-26T21:52:31.675ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
72312025-08-26T21:52:31.675ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 14 reason = Timeout
72322025-08-26T21:52:31.675ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
72332025-08-26T21:52:31.675ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 14 reason = Timeout
72342025-08-26T21:52:31.675ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
72352025-08-26T21:52:31.675ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
72362025-08-26T21:52:31.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
72372025-08-26T21:52:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
72382025-08-26T21:52:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
72392025-08-26T21:52:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
72402025-08-26T21:52:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
72412025-08-26T21:52:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
72422025-08-26T21:52:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
72432025-08-26T21:52:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
72442025-08-26T21:52:31.676ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
72452025-08-26T21:52:31.676ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.8492ms iteration = 79 status = {"chicken_switches_updated":false}
72462025-08-26T21:52:31.676ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72472025-08-26T21:52:31.677ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72482025-08-26T21:52:31.677ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72492025-08-26T21:52:31.677ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72502025-08-26T21:52:31.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72512025-08-26T21:52:31.680ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
72522025-08-26T21:52:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
72532025-08-26T21:52:31.680ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
72542025-08-26T21:52:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
72552025-08-26T21:52:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
72562025-08-26T21:52:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
72572025-08-26T21:52:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
72582025-08-26T21:52:31.681ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
72592025-08-26T21:52:31.681ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
72602025-08-26T21:52:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
72612025-08-26T21:52:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
72622025-08-26T21:52:31.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
72632025-08-26T21:52:31.682ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
72642025-08-26T21:52:31.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
72652025-08-26T21:52:31.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
72662025-08-26T21:52:31.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
72672025-08-26T21:52:31.682ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 7.248204ms iteration = 14 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
72682025-08-26T21:52:31.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
72692025-08-26T21:52:31.683ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1255, tv_nsec: 541238284 } })
72702025-08-26T21:52:31.683ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
72712025-08-26T21:52:31.683ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
72722025-08-26T21:52:31.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
72732025-08-26T21:52:31.683ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
72742025-08-26T21:52:31.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
72752025-08-26T21:52:31.683ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
72762025-08-26T21:52:31.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
72772025-08-26T21:52:31.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
72782025-08-26T21:52:31.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
72792025-08-26T21:52:31.684ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72802025-08-26T21:52:31.685ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72812025-08-26T21:52:31.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72822025-08-26T21:52:31.686ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 14 reason = Timeout
72832025-08-26T21:52:31.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 40 reason = Timeout
72842025-08-26T21:52:31.686ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
72852025-08-26T21:52:31.686ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
72862025-08-26T21:52:31.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
72872025-08-26T21:52:31.686ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 14 reason = Timeout
72882025-08-26T21:52:31.686ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
72892025-08-26T21:52:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
72902025-08-26T21:52:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
72912025-08-26T21:52:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
72922025-08-26T21:52:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
72932025-08-26T21:52:31.687ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
72942025-08-26T21:52:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
72952025-08-26T21:52:31.687ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
72962025-08-26T21:52:31.687ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
72972025-08-26T21:52:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
72982025-08-26T21:52:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
72992025-08-26T21:52:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
73002025-08-26T21:52:31.688ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 14 reason = Timeout
73012025-08-26T21:52:31.688ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 14 reason = Timeout
73022025-08-26T21:52:31.688ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 14 reason = Timeout
73032025-08-26T21:52:31.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
73042025-08-26T21:52:31.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
73052025-08-26T21:52:31.688ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 12.02\u{b5}s iteration = 14 status = {"disabled":true,"errors":[],"sps":[]}
73062025-08-26T21:52:31.688ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 14 reason = Timeout
73072025-08-26T21:52:31.688ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 14 reason = Timeout
73082025-08-26T21:52:31.688ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
73092025-08-26T21:52:31.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
73102025-08-26T21:52:31.688ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 14 reason = Timeout
73112025-08-26T21:52:31.688ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
73122025-08-26T21:52:31.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
73132025-08-26T21:52:31.688ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 14 reason = Timeout
73142025-08-26T21:52:31.688ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
73152025-08-26T21:52:31.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
73162025-08-26T21:52:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
73172025-08-26T21:52:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
73182025-08-26T21:52:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
73192025-08-26T21:52:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
73202025-08-26T21:52:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
73212025-08-26T21:52:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
73222025-08-26T21:52:31.688ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73232025-08-26T21:52:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
73242025-08-26T21:52:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
73252025-08-26T21:52:31.689ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73262025-08-26T21:52:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
73272025-08-26T21:52:31.689ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
73282025-08-26T21:52:31.689ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 14.436705ms iteration = 14 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
73292025-08-26T21:52:31.689ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
73302025-08-26T21:52:31.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
73312025-08-26T21:52:31.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 2.387444ms iteration = 14 status = {"errors":[],"finish_invoked_ok":[]}
73322025-08-26T21:52:31.690ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73332025-08-26T21:52:31.691ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
73342025-08-26T21:52:31.691ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
73352025-08-26T21:52:31.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
73362025-08-26T21:52:31.692ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
73372025-08-26T21:52:31.692ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
73382025-08-26T21:52:31.692ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
73392025-08-26T21:52:31.692ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
73402025-08-26T21:52:31.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
73412025-08-26T21:52:31.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
73422025-08-26T21:52:31.694ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
73432025-08-26T21:52:31.694ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
73442025-08-26T21:52:31.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
73452025-08-26T21:52:31.695ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
73462025-08-26T21:52:31.695ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
73472025-08-26T21:52:31.695ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
73482025-08-26T21:52:31.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
73492025-08-26T21:52:31.696ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73502025-08-26T21:52:31.699ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
73512025-08-26T21:52:31.699ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 12.920591ms iteration = 14 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
73522025-08-26T21:52:31.700ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
73532025-08-26T21:52:31.700ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
73542025-08-26T21:52:31.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
73552025-08-26T21:52:31.720ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
73562025-08-26T21:52:31.720ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 44.963854ms iteration = 40 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
73572025-08-26T21:52:31.752ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 77.101238ms iteration = 15 status = {"physical_disks_added":0}
73582025-08-26T21:52:31.753ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 65.177209ms iteration = 14 status = {"errors":[],"garbage_collect_requested":[]}
73592025-08-26T21:52:31.753ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
73602025-08-26T21:52:31.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
73612025-08-26T21:52:31.756ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 68.203117ms iteration = 14 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
73622025-08-26T21:52:31.756ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
73632025-08-26T21:52:31.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
73642025-08-26T21:52:31.757ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 70.764155ms iteration = 40 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
73652025-08-26T21:52:31.758ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 70.239273ms iteration = 14 status = {"errors":[],"requests_created_ok":[]}
73662025-08-26T21:52:31.758ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
73672025-08-26T21:52:31.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
73682025-08-26T21:52:31.759ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73692025-08-26T21:52:31.760ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 72.33111ms iteration = 14 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
73702025-08-26T21:52:31.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
73712025-08-26T21:52:31.762ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
73722025-08-26T21:52:31.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
73732025-08-26T21:52:31.762ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
73742025-08-26T21:52:31.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
73752025-08-26T21:52:31.763ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 88.672509ms iteration = 14 status = 0
73762025-08-26T21:52:31.764ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
73772025-08-26T21:52:31.764ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
73782025-08-26T21:52:31.764ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 77.323443ms iteration = 14 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
73792025-08-26T21:52:31.764ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73802025-08-26T21:52:31.766ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
73812025-08-26T21:52:31.766ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
73822025-08-26T21:52:31.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
73832025-08-26T21:52:31.767ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73842025-08-26T21:52:31.769ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
73852025-08-26T21:52:31.769ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
73862025-08-26T21:52:31.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
73872025-08-26T21:52:31.774ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
73882025-08-26T21:52:31.774ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
73892025-08-26T21:52:31.775ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d2dadbd-8d62-4790-b3cf-b8d5be369784", "content-length": "26", "date": "Tue, 26 Aug 2025 21:52:31 GMT"} })
73902025-08-26T21:52:31.775ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
73912025-08-26T21:52:31.776ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
73922025-08-26T21:52:31.782ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
73932025-08-26T21:52:31.782ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
73942025-08-26T21:52:31.782ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d5fb1b9-5f37-41b3-b094-1e172358fb3d", "content-length": "26", "date": "Tue, 26 Aug 2025 21:52:31 GMT"} })
73952025-08-26T21:52:31.782ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
73962025-08-26T21:52:31.784ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
73972025-08-26T21:52:31.795ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
73982025-08-26T21:52:31.795ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
73992025-08-26T21:52:31.796ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c974347-31fe-4072-a93c-6a3ebd889d2c", "content-length": "26", "date": "Tue, 26 Aug 2025 21:52:31 GMT"} })
74002025-08-26T21:52:31.796ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
74012025-08-26T21:52:31.799ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
74022025-08-26T21:52:31.819ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
74032025-08-26T21:52:31.819ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
74042025-08-26T21:52:31.819ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa6089f2-6ed8-40d9-83af-a036af550079", "content-length": "372", "date": "Tue, 26 Aug 2025 21:52:31 GMT"} })
74052025-08-26T21:52:31.819ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
74062025-08-26T21:52:31.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
74072025-08-26T21:52:31.838ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
74082025-08-26T21:52:31.838ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
74092025-08-26T21:52:31.838ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "67351891-adb1-4e11-957c-159a0f9a92a4", "date": "Tue, 26 Aug 2025 21:52:31 GMT"} })
74102025-08-26T21:52:31.838ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
74112025-08-26T21:52:31.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
74122025-08-26T21:52:31.841ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
74132025-08-26T21:52:31.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
74142025-08-26T21:52:31.846ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
74152025-08-26T21:52:31.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
74162025-08-26T21:52:31.851ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
74172025-08-26T21:52:31.851ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
74182025-08-26T21:52:31.851ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "a8c0ea37-f867-4494-9696-f9ddfa23c007", "date": "Tue, 26 Aug 2025 21:52:31 GMT"} })
74192025-08-26T21:52:31.852ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 164.899047ms iteration = 14 status = {}
74202025-08-26T21:52:31.891ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
74212025-08-26T21:52:31.892ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c953db5-6676-46b4-b539-235ba3d479de", "content-length": "691", "date": "Tue, 26 Aug 2025 21:52:31 GMT"} })
74222025-08-26T21:52:31.892ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
74232025-08-26T21:52:31.892ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
74242025-08-26T21:52:31.892ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
74252025-08-26T21:52:31.893ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 218.610898ms iteration = 14 status = {}
74262025-08-26T21:52:31.897ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
74272025-08-26T21:52:31.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
74282025-08-26T21:52:31.900ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 235.006658ms iteration = 14 status = null
74292025-08-26T21:52:31.970ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 305.452535ms iteration = 14 status = {}
74302025-08-26T21:52:32.034ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74312025-08-26T21:52:32.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74322025-08-26T21:52:32.037ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
74332025-08-26T21:52:32.037ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74342025-08-26T21:52:32.038ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74352025-08-26T21:52:32.049ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
74362025-08-26T21:52:32.049ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
74372025-08-26T21:52:32.050ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
74382025-08-26T21:52:32.050ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
74392025-08-26T21:52:32.050ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
74402025-08-26T21:52:32.050ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
74412025-08-26T21:52:32.050ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
74422025-08-26T21:52:32.050ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
74432025-08-26T21:52:32.050ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
74442025-08-26T21:52:32.050ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
74452025-08-26T21:52:32.050ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
74462025-08-26T21:52:32.050ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74472025-08-26T21:52:32.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74482025-08-26T21:52:32.052ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
74492025-08-26T21:52:32.052ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
74502025-08-26T21:52:32.193ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "87157877-b6fb-4549-9804-425ea0716654", "date": "Tue, 26 Aug 2025 21:52:31 GMT"} })
74512025-08-26T21:52:32.193ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
74522025-08-26T21:52:32.193ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
74532025-08-26T21:52:32.198ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "be0138b0-320a-4f15-9568-5c3176a69d6f", "content-length": "682", "date": "Tue, 26 Aug 2025 21:52:32 GMT"} })
74542025-08-26T21:52:32.198ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74552025-08-26T21:52:32.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74562025-08-26T21:52:32.203ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74572025-08-26T21:52:32.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74582025-08-26T21:52:32.206ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74592025-08-26T21:52:32.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74602025-08-26T21:52:32.209ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
74612025-08-26T21:52:32.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
74622025-08-26T21:52:32.216ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 541.115938ms iteration = 14 status = {}
74632025-08-26T21:52:35.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 341 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 609bdd64-ff85-4011-8fa6-901fbe6609e6 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
74642025-08-26T21:52:36.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 80 reason = Timeout
74652025-08-26T21:52:36.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
74662025-08-26T21:52:36.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
74672025-08-26T21:52:36.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74682025-08-26T21:52:36.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
74692025-08-26T21:52:36.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
74702025-08-26T21:52:36.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
74712025-08-26T21:52:36.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
74722025-08-26T21:52:36.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.472486ms iteration = 80 status = {"chicken_switches_updated":false}
74732025-08-26T21:52:41.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 81 reason = Timeout
74742025-08-26T21:52:41.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
74752025-08-26T21:52:41.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
74762025-08-26T21:52:41.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74772025-08-26T21:52:41.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
74782025-08-26T21:52:41.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
74792025-08-26T21:52:41.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
74802025-08-26T21:52:41.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
74812025-08-26T21:52:41.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.286924ms iteration = 81 status = {"chicken_switches_updated":false}
74822025-08-26T21:52:41.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
74832025-08-26T21:52:41.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74842025-08-26T21:52:41.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74852025-08-26T21:52:41.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74862025-08-26T21:52:41.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
74872025-08-26T21:52:41.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74882025-08-26T21:52:41.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74892025-08-26T21:52:41.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74902025-08-26T21:52:41.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74912025-08-26T21:52:41.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74922025-08-26T21:52:41.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
74932025-08-26T21:52:41.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74942025-08-26T21:52:41.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
74952025-08-26T21:52:41.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 41 reason = Timeout
74962025-08-26T21:52:41.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
74972025-08-26T21:52:41.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
74982025-08-26T21:52:41.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
74992025-08-26T21:52:41.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
75002025-08-26T21:52:41.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.935468ms iteration = 41 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
75012025-08-26T21:52:41.700ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
75022025-08-26T21:52:41.700ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.950805ms iteration = 41 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
75032025-08-26T21:52:45.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = e2a7e4bd-321b-4878-8596-e31cb3447d2e roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
75042025-08-26T21:52:45.345ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = e2a7e4bd-321b-4878-8596-e31cb3447d2e resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
75052025-08-26T21:52:45.347ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 1988 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = e2a7e4bd-321b-4878-8596-e31cb3447d2e response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
75062025-08-26T21:52:45.347ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 144d7d44-aa91-4c53-bf82-f530b6cf2629 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
75072025-08-26T21:52:45.348ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 144d7d44-aa91-4c53-bf82-f530b6cf2629 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
75082025-08-26T21:52:45.349ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2298 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 144d7d44-aa91-4c53-bf82-f530b6cf2629 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
75092025-08-26T21:52:45.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 305 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = b65b7282-6f2b-48af-816c-345a8c96d232 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
75102025-08-26T21:52:46.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 82 reason = Timeout
75112025-08-26T21:52:46.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
75122025-08-26T21:52:46.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
75132025-08-26T21:52:46.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75142025-08-26T21:52:46.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
75152025-08-26T21:52:46.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
75162025-08-26T21:52:46.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
75172025-08-26T21:52:46.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
75182025-08-26T21:52:46.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.339555ms iteration = 82 status = {"chicken_switches_updated":false}
75192025-08-26T21:52:51.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 83 reason = Timeout
75202025-08-26T21:52:51.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
75212025-08-26T21:52:51.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
75222025-08-26T21:52:51.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75232025-08-26T21:52:51.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
75242025-08-26T21:52:51.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
75252025-08-26T21:52:51.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
75262025-08-26T21:52:51.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
75272025-08-26T21:52:51.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.55406ms iteration = 83 status = {"chicken_switches_updated":false}
75282025-08-26T21:52:51.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
75292025-08-26T21:52:51.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75302025-08-26T21:52:51.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75312025-08-26T21:52:51.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75322025-08-26T21:52:51.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
75332025-08-26T21:52:51.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75342025-08-26T21:52:51.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75352025-08-26T21:52:51.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75362025-08-26T21:52:51.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75372025-08-26T21:52:51.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75382025-08-26T21:52:51.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
75392025-08-26T21:52:51.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75402025-08-26T21:52:51.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
75412025-08-26T21:52:51.685ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 42 reason = Timeout
75422025-08-26T21:52:51.685ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
75432025-08-26T21:52:51.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
75442025-08-26T21:52:51.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
75452025-08-26T21:52:51.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
75462025-08-26T21:52:51.689ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.517929ms iteration = 42 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
75472025-08-26T21:52:51.700ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
75482025-08-26T21:52:51.700ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.340872ms iteration = 42 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
75492025-08-26T21:52:55.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 409 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = d5bd21c4-df9f-4c13-b453-3efb0e693c4c response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
75502025-08-26T21:52:56.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 84 reason = Timeout
75512025-08-26T21:52:56.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
75522025-08-26T21:52:56.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
75532025-08-26T21:52:56.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75542025-08-26T21:52:56.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
75552025-08-26T21:52:56.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
75562025-08-26T21:52:56.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
75572025-08-26T21:52:56.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
75582025-08-26T21:52:56.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.061222ms iteration = 84 status = {"chicken_switches_updated":false}
75592025-08-26T21:53:00.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = bc28c7a1-c64d-4b58-a5c7-65459a31720a roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
75602025-08-26T21:53:00.345ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = bc28c7a1-c64d-4b58-a5c7-65459a31720a resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
75612025-08-26T21:53:00.348ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3164 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = bc28c7a1-c64d-4b58-a5c7-65459a31720a response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
75622025-08-26T21:53:00.348ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = f254df11-7f53-4e92-a64f-85c336b1c9e8 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
75632025-08-26T21:53:00.348ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = f254df11-7f53-4e92-a64f-85c336b1c9e8 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
75642025-08-26T21:53:00.350ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 1571 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = f254df11-7f53-4e92-a64f-85c336b1c9e8 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
75652025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
75662025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
75672025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 8 reason = Timeout
75682025-08-26T21:53:01.664ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:43:01.664253359 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
75692025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
75702025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
75712025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
75722025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
75732025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
75742025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
75752025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
75762025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
75772025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
75782025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
75792025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
75802025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
75812025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
75822025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 15 reason = Timeout
75832025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
75842025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
75852025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
75862025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
75872025-08-26T21:53:01.664ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
75882025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 15\u{b5}s iteration = 9 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
75892025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
75902025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
75912025-08-26T21:53:01.664ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1285, tv_nsec: 523688638 } })
75922025-08-26T21:53:01.664ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1285, tv_nsec: 523714379 } })
75932025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
75942025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
75952025-08-26T21:53:01.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
75962025-08-26T21:53:01.665ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
75972025-08-26T21:53:01.665ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.077075ms iteration = 8 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
75982025-08-26T21:53:01.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
75992025-08-26T21:53:01.665ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
76002025-08-26T21:53:01.665ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.134906ms iteration = 8 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
76012025-08-26T21:53:01.665ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 85 reason = Timeout
76022025-08-26T21:53:01.665ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
76032025-08-26T21:53:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
76042025-08-26T21:53:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
76052025-08-26T21:53:01.666ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76062025-08-26T21:53:01.667ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
76072025-08-26T21:53:01.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
76082025-08-26T21:53:01.669ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
76092025-08-26T21:53:01.669ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
76102025-08-26T21:53:01.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
76112025-08-26T21:53:01.669ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76122025-08-26T21:53:01.670ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:43:01.664253359 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
76132025-08-26T21:53:01.670ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 6.364064ms iteration = 8 status = {"expiration":"2025-08-26T21:43:01.664253359Z","pruned":[]}
76142025-08-26T21:53:01.670ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1285, tv_nsec: 529527000 } })
76152025-08-26T21:53:01.670ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1285, tv_nsec: 529571911 } })
76162025-08-26T21:53:01.670ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
76172025-08-26T21:53:01.671ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
76182025-08-26T21:53:01.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76192025-08-26T21:53:01.671ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76202025-08-26T21:53:01.672ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
76212025-08-26T21:53:01.672ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 10 reason = Timeout
76222025-08-26T21:53:01.672ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
76232025-08-26T21:53:01.672ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 13.661\u{b5}s iteration = 10 status = "Disabled"
76242025-08-26T21:53:01.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
76252025-08-26T21:53:01.673ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 8.86908ms iteration = 15 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
76262025-08-26T21:53:01.674ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 8 reason = Timeout
76272025-08-26T21:53:01.674ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
76282025-08-26T21:53:01.674ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
76292025-08-26T21:53:01.674ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 15 reason = Timeout
76302025-08-26T21:53:01.674ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
76312025-08-26T21:53:01.674ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
76322025-08-26T21:53:01.674ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
76332025-08-26T21:53:01.674ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
76342025-08-26T21:53:01.674ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
76352025-08-26T21:53:01.674ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 15 reason = Timeout
76362025-08-26T21:53:01.674ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 16 reason = Timeout
76372025-08-26T21:53:01.674ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
76382025-08-26T21:53:01.674ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
76392025-08-26T21:53:01.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
76402025-08-26T21:53:01.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76412025-08-26T21:53:01.674ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 15 reason = Timeout
76422025-08-26T21:53:01.674ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
76432025-08-26T21:53:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
76442025-08-26T21:53:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
76452025-08-26T21:53:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
76462025-08-26T21:53:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
76472025-08-26T21:53:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
76482025-08-26T21:53:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
76492025-08-26T21:53:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
76502025-08-26T21:53:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
76512025-08-26T21:53:01.675ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76522025-08-26T21:53:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
76532025-08-26T21:53:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
76542025-08-26T21:53:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
76552025-08-26T21:53:01.676ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
76562025-08-26T21:53:01.676ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76572025-08-26T21:53:01.676ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76582025-08-26T21:53:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
76592025-08-26T21:53:01.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76602025-08-26T21:53:01.676ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76612025-08-26T21:53:01.676ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76622025-08-26T21:53:01.677ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
76632025-08-26T21:53:01.677ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
76642025-08-26T21:53:01.677ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
76652025-08-26T21:53:01.677ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
76662025-08-26T21:53:01.677ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
76672025-08-26T21:53:01.677ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
76682025-08-26T21:53:01.677ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
76692025-08-26T21:53:01.677ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
76702025-08-26T21:53:01.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
76712025-08-26T21:53:01.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
76722025-08-26T21:53:01.678ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
76732025-08-26T21:53:01.678ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 12.669885ms iteration = 85 status = {"chicken_switches_updated":false}
76742025-08-26T21:53:01.679ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76752025-08-26T21:53:01.679ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76762025-08-26T21:53:01.680ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
76772025-08-26T21:53:01.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
76782025-08-26T21:53:01.681ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
76792025-08-26T21:53:01.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
76802025-08-26T21:53:01.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
76812025-08-26T21:53:01.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
76822025-08-26T21:53:01.681ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
76832025-08-26T21:53:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
76842025-08-26T21:53:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
76852025-08-26T21:53:01.682ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
76862025-08-26T21:53:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
76872025-08-26T21:53:01.682ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
76882025-08-26T21:53:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
76892025-08-26T21:53:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
76902025-08-26T21:53:01.683ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 9.044674ms iteration = 15 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
76912025-08-26T21:53:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
76922025-08-26T21:53:01.683ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
76932025-08-26T21:53:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
76942025-08-26T21:53:01.683ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
76952025-08-26T21:53:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
76962025-08-26T21:53:01.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76972025-08-26T21:53:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
76982025-08-26T21:53:01.684ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
76992025-08-26T21:53:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
77002025-08-26T21:53:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77012025-08-26T21:53:01.684ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
77022025-08-26T21:53:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
77032025-08-26T21:53:01.684ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
77042025-08-26T21:53:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
77052025-08-26T21:53:01.684ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
77062025-08-26T21:53:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
77072025-08-26T21:53:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
77082025-08-26T21:53:01.685ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77092025-08-26T21:53:01.685ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 15 reason = Timeout
77102025-08-26T21:53:01.685ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 8 reason = Timeout
77112025-08-26T21:53:01.685ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
77122025-08-26T21:53:01.685ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
77132025-08-26T21:53:01.685ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 15 reason = Timeout
77142025-08-26T21:53:01.685ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
77152025-08-26T21:53:01.685ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
77162025-08-26T21:53:01.685ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 43 reason = Timeout
77172025-08-26T21:53:01.685ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
77182025-08-26T21:53:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
77192025-08-26T21:53:01.686ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
77202025-08-26T21:53:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
77212025-08-26T21:53:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
77222025-08-26T21:53:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
77232025-08-26T21:53:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
77242025-08-26T21:53:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
77252025-08-26T21:53:01.686ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77262025-08-26T21:53:01.686ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
77272025-08-26T21:53:01.686ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 0 version = 2
77282025-08-26T21:53:01.686ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
77292025-08-26T21:53:01.686ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
77302025-08-26T21:53:01.686ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 15 reason = Timeout
77312025-08-26T21:53:01.686ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
77322025-08-26T21:53:01.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
77332025-08-26T21:53:01.686ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77342025-08-26T21:53:01.686ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 8 reason = Timeout
77352025-08-26T21:53:01.686ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
77362025-08-26T21:53:01.686ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 8 reason = Timeout
77372025-08-26T21:53:01.686ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
77382025-08-26T21:53:01.686ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 15 reason = Timeout
77392025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
77402025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 8 reason = Timeout
77412025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
77422025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 8 reason = Timeout
77432025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
77442025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 15 reason = Timeout
77452025-08-26T21:53:01.687ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
77462025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
77472025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 15 reason = Timeout
77482025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
77492025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 15 reason = Timeout
77502025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
77512025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 15 reason = Timeout
77522025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
77532025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 15 reason = Timeout
77542025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 8.72\u{b5}s iteration = 15 status = {"disabled":true,"errors":[],"sps":[]}
77552025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
77562025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
77572025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
77582025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
77592025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
77602025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
77612025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
77622025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
77632025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
77642025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
77652025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
77662025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
77672025-08-26T21:53:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
77682025-08-26T21:53:01.689ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
77692025-08-26T21:53:01.689ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
77702025-08-26T21:53:01.689ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
77712025-08-26T21:53:01.689ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
77722025-08-26T21:53:01.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
77732025-08-26T21:53:01.689ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 2.628189ms iteration = 15 status = {"errors":[],"finish_invoked_ok":[]}
77742025-08-26T21:53:01.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
77752025-08-26T21:53:01.690ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
77762025-08-26T21:53:01.690ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
77772025-08-26T21:53:01.690ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 3.858597ms iteration = 8 status = {"errors":[],"found_port_ok":[]}
77782025-08-26T21:53:01.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
77792025-08-26T21:53:01.691ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
77802025-08-26T21:53:01.691ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 4.203754ms iteration = 8 status = {"by_rx":{},"error":null}
77812025-08-26T21:53:01.691ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
77822025-08-26T21:53:01.691ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
77832025-08-26T21:53:01.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
77842025-08-26T21:53:01.692ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
77852025-08-26T21:53:01.692ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
77862025-08-26T21:53:01.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
77872025-08-26T21:53:01.693ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
77882025-08-26T21:53:01.693ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
77892025-08-26T21:53:01.693ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 6.476736ms iteration = 8 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
77902025-08-26T21:53:01.694ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
77912025-08-26T21:53:01.694ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77922025-08-26T21:53:01.694ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77932025-08-26T21:53:01.694ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77942025-08-26T21:53:01.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
77952025-08-26T21:53:01.694ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77962025-08-26T21:53:01.697ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
77972025-08-26T21:53:01.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
77982025-08-26T21:53:01.698ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
77992025-08-26T21:53:01.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
78002025-08-26T21:53:01.699ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
78012025-08-26T21:53:01.699ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
78022025-08-26T21:53:01.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
78032025-08-26T21:53:01.699ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 25.623768ms iteration = 16 status = {"physical_disks_added":0}
78042025-08-26T21:53:01.700ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
78052025-08-26T21:53:01.700ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
78062025-08-26T21:53:01.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
78072025-08-26T21:53:01.700ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
78082025-08-26T21:53:01.700ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
78092025-08-26T21:53:01.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
78102025-08-26T21:53:01.701ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
78112025-08-26T21:53:01.701ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
78122025-08-26T21:53:01.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
78132025-08-26T21:53:01.702ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
78142025-08-26T21:53:01.702ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 15.739865ms iteration = 8 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
78152025-08-26T21:53:01.702ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-08-26T21:45:59.866217Z, time_modified: 2025-08-26T21:45:59.866217Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }) }
78162025-08-26T21:53:01.702ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 38.513058ms iteration = 8 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]}},"default_endpoint":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
78172025-08-26T21:53:01.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
78182025-08-26T21:53:01.704ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78192025-08-26T21:53:01.707ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 4 version = 2
78202025-08-26T21:53:01.707ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
78212025-08-26T21:53:01.707ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
78222025-08-26T21:53:01.707ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-08-26 21:46:05.296205 UTC dns_group = external generation = 2
78232025-08-26T21:53:01.707ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 42.931958ms iteration = 8 status = {"generation":2}
78242025-08-26T21:53:01.707ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
78252025-08-26T21:53:01.707ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
78262025-08-26T21:53:01.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
78272025-08-26T21:53:01.708ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
78282025-08-26T21:53:01.708ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 22.432246ms iteration = 15 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
78292025-08-26T21:53:01.708ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
78302025-08-26T21:53:01.708ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
78312025-08-26T21:53:01.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
78322025-08-26T21:53:01.709ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78332025-08-26T21:53:01.713ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
78342025-08-26T21:53:01.713ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
78352025-08-26T21:53:01.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
78362025-08-26T21:53:01.715ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
78372025-08-26T21:53:01.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
78382025-08-26T21:53:01.717ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 29.972296ms iteration = 15 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
78392025-08-26T21:53:01.717ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
78402025-08-26T21:53:01.717ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
78412025-08-26T21:53:01.717ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78422025-08-26T21:53:01.717ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-08-26 21:46:04.065217 UTC dns_group = internal generation = 1
78432025-08-26T21:53:01.717ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 53.313511ms iteration = 8 status = {"generation":1}
78442025-08-26T21:53:01.721ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
78452025-08-26T21:53:01.721ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
78462025-08-26T21:53:01.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
78472025-08-26T21:53:01.723ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78482025-08-26T21:53:01.726ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
78492025-08-26T21:53:01.726ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
78502025-08-26T21:53:01.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
78512025-08-26T21:53:01.731ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
78522025-08-26T21:53:01.731ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 56.694648ms iteration = 43 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
78532025-08-26T21:53:01.740ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78542025-08-26T21:53:01.744ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
78552025-08-26T21:53:01.744ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78562025-08-26T21:53:01.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78572025-08-26T21:53:01.750ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1285, tv_nsec: 609703227 } })
78582025-08-26T21:53:01.751ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
78592025-08-26T21:53:01.760ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 73.731832ms iteration = 15 status = {"errors":[],"garbage_collect_requested":[]}
78602025-08-26T21:53:01.770ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
78612025-08-26T21:53:01.770ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 95.899202ms iteration = 15 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
78622025-08-26T21:53:01.776ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
78632025-08-26T21:53:01.776ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
78642025-08-26T21:53:01.776ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
78652025-08-26T21:53:01.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
78662025-08-26T21:53:01.778ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 91.735648ms iteration = 15 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
78672025-08-26T21:53:01.779ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
78682025-08-26T21:53:01.779ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
78692025-08-26T21:53:01.779ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 105.384736ms iteration = 8 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
78702025-08-26T21:53:01.784ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 97.335795ms iteration = 15 status = {"errors":[],"requests_created_ok":[]}
78712025-08-26T21:53:01.786ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 100.24209ms iteration = 43 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
78722025-08-26T21:53:01.789ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
78732025-08-26T21:53:01.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
78742025-08-26T21:53:01.791ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
78752025-08-26T21:53:01.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
78762025-08-26T21:53:01.793ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78772025-08-26T21:53:01.797ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
78782025-08-26T21:53:01.797ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
78792025-08-26T21:53:01.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
78802025-08-26T21:53:01.799ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 125.300845ms iteration = 15 status = 0
78812025-08-26T21:53:01.832ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
78822025-08-26T21:53:01.832ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
78832025-08-26T21:53:01.832ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 146.956643ms iteration = 15 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
78842025-08-26T21:53:01.846ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
78852025-08-26T21:53:01.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
78862025-08-26T21:53:01.849ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
78872025-08-26T21:53:01.849ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 164.027907ms iteration = 8 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
78882025-08-26T21:53:01.853ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
78892025-08-26T21:53:01.857ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
78902025-08-26T21:53:01.857ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
78912025-08-26T21:53:01.857ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ffd0428a-57c7-4a70-987e-6890ec70ba1a", "content-length": "26", "date": "Tue, 26 Aug 2025 21:53:01 GMT"} })
78922025-08-26T21:53:01.857ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
78932025-08-26T21:53:01.859ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
78942025-08-26T21:53:01.932ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
78952025-08-26T21:53:01.960ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
78962025-08-26T21:53:01.961ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "735c035e-464d-43cb-87ab-8919c200c9ce", "content-length": "691", "date": "Tue, 26 Aug 2025 21:53:01 GMT"} })
78972025-08-26T21:53:01.961ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
78982025-08-26T21:53:01.961ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
78992025-08-26T21:53:01.961ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
79002025-08-26T21:53:01.963ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 288.963314ms iteration = 15 status = {}
79012025-08-26T21:53:01.968ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
79022025-08-26T21:53:01.968ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
79032025-08-26T21:53:01.968ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "42734b76-1e09-404b-bd46-1cdcf4989ccc", "content-length": "26", "date": "Tue, 26 Aug 2025 21:53:01 GMT"} })
79042025-08-26T21:53:01.968ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
79052025-08-26T21:53:01.969ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
79062025-08-26T21:53:01.987ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
79072025-08-26T21:53:01.988ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
79082025-08-26T21:53:01.988ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b00c8bac-3f47-49e9-a9c6-73b902cb4d6c", "content-length": "26", "date": "Tue, 26 Aug 2025 21:53:01 GMT"} })
79092025-08-26T21:53:01.988ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
79102025-08-26T21:53:01.990ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
79112025-08-26T21:53:02.000ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 336.194359ms iteration = 15 status = {}
79122025-08-26T21:53:02.028ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
79132025-08-26T21:53:02.028ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
79142025-08-26T21:53:02.028ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "daccf3cd-4ba8-4d7b-901a-28454f5daa2b", "content-length": "372", "date": "Tue, 26 Aug 2025 21:53:01 GMT"} })
79152025-08-26T21:53:02.028ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
79162025-08-26T21:53:02.029ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
79172025-08-26T21:53:02.035ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
79182025-08-26T21:53:02.036ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "7165ce52-d382-4d38-a392-efa9d1771df4", "date": "Tue, 26 Aug 2025 21:53:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
79192025-08-26T21:53:02.036ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
79202025-08-26T21:53:02.038ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "7f3fd0e4-b19d-4f63-85cb-a6ef67c5d4b8", "date": "Tue, 26 Aug 2025 21:53:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
79212025-08-26T21:53:02.038ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
79222025-08-26T21:53:02.040ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
79232025-08-26T21:53:02.040ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 376.384834ms iteration = 10 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
79242025-08-26T21:53:02.043ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
79252025-08-26T21:53:02.043ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
79262025-08-26T21:53:02.043ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "287bcb35-176c-41bf-a47c-02476a049eec", "date": "Tue, 26 Aug 2025 21:53:01 GMT"} })
79272025-08-26T21:53:02.043ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
79282025-08-26T21:53:02.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
79292025-08-26T21:53:02.046ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
79302025-08-26T21:53:02.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
79312025-08-26T21:53:02.049ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
79322025-08-26T21:53:02.049ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
79332025-08-26T21:53:02.052ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
79342025-08-26T21:53:02.052ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
79352025-08-26T21:53:02.052ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c541012e-6a5f-47eb-b595-ee7615c1458f", "date": "Tue, 26 Aug 2025 21:53:01 GMT"} })
79362025-08-26T21:53:02.054ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 367.170827ms iteration = 15 status = {}
79372025-08-26T21:53:02.066ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
79382025-08-26T21:53:02.080ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
79392025-08-26T21:53:02.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
79402025-08-26T21:53:02.083ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 418.684268ms iteration = 15 status = null
79412025-08-26T21:53:02.132ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
79422025-08-26T21:53:02.133ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "ce072efe-74cb-44a3-86a0-7a35cc37a191", "date": "Tue, 26 Aug 2025 21:53:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
79432025-08-26T21:53:02.133ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
79442025-08-26T21:53:02.134ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "cfb069eb-c9ba-478b-beb1-4914f9edc281", "date": "Tue, 26 Aug 2025 21:53:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
79452025-08-26T21:53:02.134ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
79462025-08-26T21:53:02.136ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "fbe974df-e9c8-408a-922c-ffd0de9cea8a", "date": "Tue, 26 Aug 2025 21:53:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
79472025-08-26T21:53:02.136ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
79482025-08-26T21:53:02.138ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
79492025-08-26T21:53:02.138ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 474.557267ms iteration = 10 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
79502025-08-26T21:53:02.167ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
79512025-08-26T21:53:02.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
79522025-08-26T21:53:02.169ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
79532025-08-26T21:53:02.169ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
79542025-08-26T21:53:02.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
79552025-08-26T21:53:02.180ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
79562025-08-26T21:53:02.180ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
79572025-08-26T21:53:02.180ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
79582025-08-26T21:53:02.180ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
79592025-08-26T21:53:02.180ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
79602025-08-26T21:53:02.180ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
79612025-08-26T21:53:02.180ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
79622025-08-26T21:53:02.180ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
79632025-08-26T21:53:02.180ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
79642025-08-26T21:53:02.180ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
79652025-08-26T21:53:02.180ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
79662025-08-26T21:53:02.180ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
79672025-08-26T21:53:02.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
79682025-08-26T21:53:02.182ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
79692025-08-26T21:53:02.182ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
79702025-08-26T21:53:02.291ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "017ef93e-7950-489f-a4e4-43b7ce3d53a9", "date": "Tue, 26 Aug 2025 21:53:01 GMT"} })
79712025-08-26T21:53:02.291ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
79722025-08-26T21:53:02.291ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
79732025-08-26T21:53:02.292ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "6133cf6a-de5e-452d-a695-2a1f51395bc0", "content-length": "682", "date": "Tue, 26 Aug 2025 21:53:01 GMT"} })
79742025-08-26T21:53:02.292ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
79752025-08-26T21:53:02.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
79762025-08-26T21:53:02.296ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
79772025-08-26T21:53:02.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
79782025-08-26T21:53:02.299ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
79792025-08-26T21:53:02.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
79802025-08-26T21:53:02.301ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
79812025-08-26T21:53:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
79822025-08-26T21:53:02.308ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 634.526763ms iteration = 15 status = {}
79832025-08-26T21:53:05.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 377 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 1c132c30-1efa-4d9d-a13a-b11fc5181b65 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
79842025-08-26T21:53:06.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 86 reason = Timeout
79852025-08-26T21:53:06.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
79862025-08-26T21:53:06.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
79872025-08-26T21:53:06.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79882025-08-26T21:53:06.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
79892025-08-26T21:53:06.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
79902025-08-26T21:53:06.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
79912025-08-26T21:53:06.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
79922025-08-26T21:53:06.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.286054ms iteration = 86 status = {"chicken_switches_updated":false}
79932025-08-26T21:53:11.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 87 reason = Timeout
79942025-08-26T21:53:11.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
79952025-08-26T21:53:11.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
79962025-08-26T21:53:11.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79972025-08-26T21:53:11.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
79982025-08-26T21:53:11.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
79992025-08-26T21:53:11.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
80002025-08-26T21:53:11.673ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
80012025-08-26T21:53:11.673ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
80022025-08-26T21:53:11.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
80032025-08-26T21:53:11.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
80042025-08-26T21:53:11.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.804896ms iteration = 87 status = {"chicken_switches_updated":false}
80052025-08-26T21:53:11.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80062025-08-26T21:53:11.678ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
80072025-08-26T21:53:11.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
80082025-08-26T21:53:11.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
80092025-08-26T21:53:11.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
80102025-08-26T21:53:11.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
80112025-08-26T21:53:11.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80122025-08-26T21:53:11.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
80132025-08-26T21:53:11.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
80142025-08-26T21:53:11.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
80152025-08-26T21:53:11.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 44 reason = Timeout
80162025-08-26T21:53:11.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
80172025-08-26T21:53:11.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
80182025-08-26T21:53:11.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
80192025-08-26T21:53:11.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
80202025-08-26T21:53:11.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.98984ms iteration = 44 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
80212025-08-26T21:53:11.698ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
80222025-08-26T21:53:11.698ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.104816ms iteration = 44 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
80232025-08-26T21:53:15.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 350 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 78ed963b-53a8-4f89-8b0c-5d9eb51115ca response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
80242025-08-26T21:53:16.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 88 reason = Timeout
80252025-08-26T21:53:16.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
80262025-08-26T21:53:16.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
80272025-08-26T21:53:16.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80282025-08-26T21:53:16.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
80292025-08-26T21:53:16.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
80302025-08-26T21:53:16.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
80312025-08-26T21:53:16.672ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
80322025-08-26T21:53:16.672ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.448995ms iteration = 88 status = {"chicken_switches_updated":false}
80332025-08-26T21:53:21.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 89 reason = Timeout
80342025-08-26T21:53:21.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
80352025-08-26T21:53:21.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
80362025-08-26T21:53:21.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80372025-08-26T21:53:21.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
80382025-08-26T21:53:21.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
80392025-08-26T21:53:21.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
80402025-08-26T21:53:21.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
80412025-08-26T21:53:21.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.933296ms iteration = 89 status = {"chicken_switches_updated":false}
80422025-08-26T21:53:21.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
80432025-08-26T21:53:21.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
80442025-08-26T21:53:21.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
80452025-08-26T21:53:21.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80462025-08-26T21:53:21.678ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
80472025-08-26T21:53:21.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
80482025-08-26T21:53:21.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
80492025-08-26T21:53:21.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
80502025-08-26T21:53:21.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
80512025-08-26T21:53:21.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80522025-08-26T21:53:21.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
80532025-08-26T21:53:21.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
80542025-08-26T21:53:21.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
80552025-08-26T21:53:21.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 45 reason = Timeout
80562025-08-26T21:53:21.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
80572025-08-26T21:53:21.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
80582025-08-26T21:53:21.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
80592025-08-26T21:53:21.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
80602025-08-26T21:53:21.689ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.081319ms iteration = 45 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
80612025-08-26T21:53:21.697ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
80622025-08-26T21:53:21.697ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.557031ms iteration = 45 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
80632025-08-26T21:53:25.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 389 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = d431d3a2-687f-47ca-85fc-359945755e49 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
80642025-08-26T21:53:26.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 90 reason = Timeout
80652025-08-26T21:53:26.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
80662025-08-26T21:53:26.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
80672025-08-26T21:53:26.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80682025-08-26T21:53:26.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
80692025-08-26T21:53:26.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
80702025-08-26T21:53:26.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
80712025-08-26T21:53:26.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
80722025-08-26T21:53:26.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.018439ms iteration = 90 status = {"chicken_switches_updated":false}
80732025-08-26T21:53:30.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = dcf640d9-e769-4767-938b-cc254733d0c7 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
80742025-08-26T21:53:30.346ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = dcf640d9-e769-4767-938b-cc254733d0c7 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
80752025-08-26T21:53:30.349ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3620 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = dcf640d9-e769-4767-938b-cc254733d0c7 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
80762025-08-26T21:53:30.349ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = a4a409c6-db4f-42dc-a082-ed3949f01ae5 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
80772025-08-26T21:53:30.349ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = a4a409c6-db4f-42dc-a082-ed3949f01ae5 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
80782025-08-26T21:53:30.352ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2704 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = a4a409c6-db4f-42dc-a082-ed3949f01ae5 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
80792025-08-26T21:53:31.665ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
80802025-08-26T21:53:31.665ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
80812025-08-26T21:53:31.665ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 16 reason = Timeout
80822025-08-26T21:53:31.665ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
80832025-08-26T21:53:31.665ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
80842025-08-26T21:53:31.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
80852025-08-26T21:53:31.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
80862025-08-26T21:53:31.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 91 reason = Timeout
80872025-08-26T21:53:31.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
80882025-08-26T21:53:31.667ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1315, tv_nsec: 525684613 } })
80892025-08-26T21:53:31.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
80902025-08-26T21:53:31.667ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
80912025-08-26T21:53:31.667ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1315, tv_nsec: 526046291 } })
80922025-08-26T21:53:31.668ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
80932025-08-26T21:53:31.668ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.834446ms iteration = 16 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
80942025-08-26T21:53:31.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80952025-08-26T21:53:31.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
80962025-08-26T21:53:31.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
80972025-08-26T21:53:31.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
80982025-08-26T21:53:31.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
80992025-08-26T21:53:31.674ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 16 reason = Timeout
81002025-08-26T21:53:31.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.172932ms iteration = 91 status = {"chicken_switches_updated":false}
81012025-08-26T21:53:31.674ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
81022025-08-26T21:53:31.674ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 16 reason = Timeout
81032025-08-26T21:53:31.674ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
81042025-08-26T21:53:31.674ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 16 reason = Timeout
81052025-08-26T21:53:31.674ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
81062025-08-26T21:53:31.674ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
81072025-08-26T21:53:31.674ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
81082025-08-26T21:53:31.674ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 17 reason = Timeout
81092025-08-26T21:53:31.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
81102025-08-26T21:53:31.674ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 16 reason = Timeout
81112025-08-26T21:53:31.674ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
81122025-08-26T21:53:31.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
81132025-08-26T21:53:31.674ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
81142025-08-26T21:53:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
81152025-08-26T21:53:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
81162025-08-26T21:53:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
81172025-08-26T21:53:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
81182025-08-26T21:53:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
81192025-08-26T21:53:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
81202025-08-26T21:53:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
81212025-08-26T21:53:31.675ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81222025-08-26T21:53:31.676ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81232025-08-26T21:53:31.676ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81242025-08-26T21:53:31.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81252025-08-26T21:53:31.676ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81262025-08-26T21:53:31.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
81272025-08-26T21:53:31.679ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
81282025-08-26T21:53:31.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
81292025-08-26T21:53:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
81302025-08-26T21:53:31.680ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
81312025-08-26T21:53:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
81322025-08-26T21:53:31.680ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
81332025-08-26T21:53:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
81342025-08-26T21:53:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
81352025-08-26T21:53:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
81362025-08-26T21:53:31.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
81372025-08-26T21:53:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
81382025-08-26T21:53:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
81392025-08-26T21:53:31.681ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
81402025-08-26T21:53:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
81412025-08-26T21:53:31.681ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1315, tv_nsec: 540207131 } })
81422025-08-26T21:53:31.682ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
81432025-08-26T21:53:31.682ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
81442025-08-26T21:53:31.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
81452025-08-26T21:53:31.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
81462025-08-26T21:53:31.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
81472025-08-26T21:53:31.683ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
81482025-08-26T21:53:31.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
81492025-08-26T21:53:31.684ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81502025-08-26T21:53:31.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81512025-08-26T21:53:31.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
81522025-08-26T21:53:31.686ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
81532025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
81542025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 16 reason = Timeout
81552025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 16 reason = Timeout
81562025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
81572025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
81582025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 46 reason = Timeout
81592025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
81602025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 16 reason = Timeout
81612025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
81622025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 16 reason = Timeout
81632025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
81642025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 16 reason = Timeout
81652025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 9.151\u{b5}s iteration = 16 status = {"disabled":true,"errors":[],"sps":[]}
81662025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 16 reason = Timeout
81672025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 16 reason = Timeout
81682025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
81692025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 16 reason = Timeout
81702025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
81712025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
81722025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
81732025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 16 reason = Timeout
81742025-08-26T21:53:31.687ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
81752025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
81762025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
81772025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
81782025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
81792025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
81802025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
81812025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
81822025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
81832025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
81842025-08-26T21:53:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
81852025-08-26T21:53:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
81862025-08-26T21:53:31.688ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
81872025-08-26T21:53:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
81882025-08-26T21:53:31.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
81892025-08-26T21:53:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
81902025-08-26T21:53:31.689ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81912025-08-26T21:53:31.689ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81922025-08-26T21:53:31.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.250783ms iteration = 46 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
81932025-08-26T21:53:31.691ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
81942025-08-26T21:53:31.691ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
81952025-08-26T21:53:31.691ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
81962025-08-26T21:53:31.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
81972025-08-26T21:53:31.693ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
81982025-08-26T21:53:31.693ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
81992025-08-26T21:53:31.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
82002025-08-26T21:53:31.693ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
82012025-08-26T21:53:31.693ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
82022025-08-26T21:53:31.693ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
82032025-08-26T21:53:31.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
82042025-08-26T21:53:31.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
82052025-08-26T21:53:31.695ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 7.950719ms iteration = 16 status = {"errors":[],"requests_created_ok":[]}
82062025-08-26T21:53:31.695ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82072025-08-26T21:53:31.695ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 8.522263ms iteration = 16 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
82082025-08-26T21:53:31.698ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
82092025-08-26T21:53:31.698ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
82102025-08-26T21:53:31.698ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
82112025-08-26T21:53:31.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
82122025-08-26T21:53:31.699ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
82132025-08-26T21:53:31.699ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
82142025-08-26T21:53:31.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
82152025-08-26T21:53:31.702ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
82162025-08-26T21:53:31.702ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 14.743562ms iteration = 16 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
82172025-08-26T21:53:31.727ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
82182025-08-26T21:53:31.727ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 53.211819ms iteration = 46 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
82192025-08-26T21:53:31.738ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
82202025-08-26T21:53:31.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
82212025-08-26T21:53:31.741ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 67.194034ms iteration = 16 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
82222025-08-26T21:53:31.742ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
82232025-08-26T21:53:31.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
82242025-08-26T21:53:31.747ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
82252025-08-26T21:53:31.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
82262025-08-26T21:53:31.749ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82272025-08-26T21:53:31.750ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 75.810978ms iteration = 17 status = {"physical_disks_added":0}
82282025-08-26T21:53:31.752ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 65.19441ms iteration = 16 status = {"errors":[],"garbage_collect_requested":[]}
82292025-08-26T21:53:31.752ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
82302025-08-26T21:53:31.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
82312025-08-26T21:53:31.754ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
82322025-08-26T21:53:31.754ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
82332025-08-26T21:53:31.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
82342025-08-26T21:53:31.754ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82352025-08-26T21:53:31.755ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
82362025-08-26T21:53:31.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
82372025-08-26T21:53:31.756ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
82382025-08-26T21:53:31.756ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
82392025-08-26T21:53:31.757ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 82.699034ms iteration = 16 status = 0
82402025-08-26T21:53:31.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
82412025-08-26T21:53:31.758ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
82422025-08-26T21:53:31.758ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 84.163527ms iteration = 16 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
82432025-08-26T21:53:31.760ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
82442025-08-26T21:53:31.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
82452025-08-26T21:53:31.764ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
82462025-08-26T21:53:31.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
82472025-08-26T21:53:31.765ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
82482025-08-26T21:53:31.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
82492025-08-26T21:53:31.767ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 79.830919ms iteration = 16 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
82502025-08-26T21:53:31.767ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
82512025-08-26T21:53:31.767ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
82522025-08-26T21:53:31.767ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 79.82523ms iteration = 16 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
82532025-08-26T21:53:31.768ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82542025-08-26T21:53:31.770ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
82552025-08-26T21:53:31.770ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
82562025-08-26T21:53:31.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
82572025-08-26T21:53:31.771ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 84.030214ms iteration = 16 status = {"errors":[],"finish_invoked_ok":[]}
82582025-08-26T21:53:31.772ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82592025-08-26T21:53:31.775ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
82602025-08-26T21:53:31.775ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
82612025-08-26T21:53:31.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
82622025-08-26T21:53:31.805ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
82632025-08-26T21:53:31.805ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
82642025-08-26T21:53:31.805ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "48f99a0d-2a42-4ef2-ae0e-acfb7bd0bc70", "content-length": "26", "date": "Tue, 26 Aug 2025 21:53:31 GMT"} })
82652025-08-26T21:53:31.805ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
82662025-08-26T21:53:31.806ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
82672025-08-26T21:53:31.820ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
82682025-08-26T21:53:31.820ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
82692025-08-26T21:53:31.821ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "d77693fd-c1e3-4fbb-96dd-5883c7f3dbee", "content-length": "372", "date": "Tue, 26 Aug 2025 21:53:31 GMT"} })
82702025-08-26T21:53:31.821ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
82712025-08-26T21:53:31.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
82722025-08-26T21:53:31.837ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
82732025-08-26T21:53:31.837ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
82742025-08-26T21:53:31.837ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "37648769-13f7-4750-8be5-31d78ca051e3", "date": "Tue, 26 Aug 2025 21:53:31 GMT"} })
82752025-08-26T21:53:31.837ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
82762025-08-26T21:53:31.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
82772025-08-26T21:53:31.840ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
82782025-08-26T21:53:31.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
82792025-08-26T21:53:31.843ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
82802025-08-26T21:53:31.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
82812025-08-26T21:53:31.846ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
82822025-08-26T21:53:31.846ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
82832025-08-26T21:53:31.846ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c4e2d79a-5151-4e26-a7e4-28c60c827308", "date": "Tue, 26 Aug 2025 21:53:31 GMT"} })
82842025-08-26T21:53:31.848ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 161.118932ms iteration = 16 status = {}
82852025-08-26T21:53:31.868ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
82862025-08-26T21:53:31.868ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
82872025-08-26T21:53:31.869ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1911cd2-6f17-4932-a144-874370a506a4", "content-length": "26", "date": "Tue, 26 Aug 2025 21:53:31 GMT"} })
82882025-08-26T21:53:31.869ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
82892025-08-26T21:53:31.871ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
82902025-08-26T21:53:31.893ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
82912025-08-26T21:53:31.893ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
82922025-08-26T21:53:31.894ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee8ee4bf-672e-40b6-86d2-506e1cd4b4c4", "content-length": "26", "date": "Tue, 26 Aug 2025 21:53:31 GMT"} })
82932025-08-26T21:53:31.894ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
82942025-08-26T21:53:31.897ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
82952025-08-26T21:53:31.898ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
82962025-08-26T21:53:31.899ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b875aa3-65c0-42b7-b126-bf61a94f4a00", "content-length": "691", "date": "Tue, 26 Aug 2025 21:53:31 GMT"} })
82972025-08-26T21:53:31.899ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
82982025-08-26T21:53:31.899ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
82992025-08-26T21:53:31.899ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
83002025-08-26T21:53:31.900ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 226.511846ms iteration = 16 status = {}
83012025-08-26T21:53:31.957ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 292.150446ms iteration = 16 status = {}
83022025-08-26T21:53:31.966ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
83032025-08-26T21:53:31.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
83042025-08-26T21:53:31.968ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 303.179544ms iteration = 16 status = null
83052025-08-26T21:53:32.004ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
83062025-08-26T21:53:32.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
83072025-08-26T21:53:32.006ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
83082025-08-26T21:53:32.006ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
83092025-08-26T21:53:32.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
83102025-08-26T21:53:32.017ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
83112025-08-26T21:53:32.017ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
83122025-08-26T21:53:32.018ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
83132025-08-26T21:53:32.018ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
83142025-08-26T21:53:32.018ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
83152025-08-26T21:53:32.018ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
83162025-08-26T21:53:32.018ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
83172025-08-26T21:53:32.018ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
83182025-08-26T21:53:32.018ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
83192025-08-26T21:53:32.018ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
83202025-08-26T21:53:32.018ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
83212025-08-26T21:53:32.018ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
83222025-08-26T21:53:32.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
83232025-08-26T21:53:32.020ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
83242025-08-26T21:53:32.020ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
83252025-08-26T21:53:32.135ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "230668ed-dce8-43ee-92dc-af5e281cdfdb", "date": "Tue, 26 Aug 2025 21:53:32 GMT"} })
83262025-08-26T21:53:32.135ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
83272025-08-26T21:53:32.135ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
83282025-08-26T21:53:32.141ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "79ecbaf1-0fdc-4b50-8b41-e30496dfa54c", "content-length": "682", "date": "Tue, 26 Aug 2025 21:53:31 GMT"} })
83292025-08-26T21:53:32.142ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
83302025-08-26T21:53:32.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
83312025-08-26T21:53:32.145ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
83322025-08-26T21:53:32.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
83332025-08-26T21:53:32.149ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
83342025-08-26T21:53:32.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
83352025-08-26T21:53:32.151ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
83362025-08-26T21:53:32.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
83372025-08-26T21:53:32.158ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 484.288897ms iteration = 16 status = {}
83382025-08-26T21:53:35.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 347 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = e98fbd41-b907-4bc1-895a-96a3007b119b response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
83392025-08-26T21:53:36.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 92 reason = Timeout
83402025-08-26T21:53:36.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
83412025-08-26T21:53:36.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
83422025-08-26T21:53:36.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83432025-08-26T21:53:36.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
83442025-08-26T21:53:36.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
83452025-08-26T21:53:36.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
83462025-08-26T21:53:36.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
83472025-08-26T21:53:36.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.615709ms iteration = 92 status = {"chicken_switches_updated":false}
83482025-08-26T21:53:41.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 93 reason = Timeout
83492025-08-26T21:53:41.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
83502025-08-26T21:53:41.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
83512025-08-26T21:53:41.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83522025-08-26T21:53:41.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
83532025-08-26T21:53:41.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
83542025-08-26T21:53:41.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
83552025-08-26T21:53:41.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
83562025-08-26T21:53:41.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
83572025-08-26T21:53:41.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
83582025-08-26T21:53:41.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
83592025-08-26T21:53:41.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.795465ms iteration = 93 status = {"chicken_switches_updated":false}
83602025-08-26T21:53:41.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83612025-08-26T21:53:41.678ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
83622025-08-26T21:53:41.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
83632025-08-26T21:53:41.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
83642025-08-26T21:53:41.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
83652025-08-26T21:53:41.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
83662025-08-26T21:53:41.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83672025-08-26T21:53:41.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
83682025-08-26T21:53:41.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
83692025-08-26T21:53:41.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
83702025-08-26T21:53:41.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 47 reason = Timeout
83712025-08-26T21:53:41.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
83722025-08-26T21:53:41.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
83732025-08-26T21:53:41.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
83742025-08-26T21:53:41.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
83752025-08-26T21:53:41.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.703404ms iteration = 47 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
83762025-08-26T21:53:41.700ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
83772025-08-26T21:53:41.700ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.744561ms iteration = 47 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
83782025-08-26T21:53:45.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 326 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = ee1e6185-f911-4fea-a30f-32487c285794 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
83792025-08-26T21:53:46.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 94 reason = Timeout
83802025-08-26T21:53:46.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
83812025-08-26T21:53:46.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
83822025-08-26T21:53:46.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83832025-08-26T21:53:46.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
83842025-08-26T21:53:46.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
83852025-08-26T21:53:46.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
83862025-08-26T21:53:46.672ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
83872025-08-26T21:53:46.672ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.032287ms iteration = 94 status = {"chicken_switches_updated":false}
83882025-08-26T21:53:51.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 95 reason = Timeout
83892025-08-26T21:53:51.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
83902025-08-26T21:53:51.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
83912025-08-26T21:53:51.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83922025-08-26T21:53:51.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
83932025-08-26T21:53:51.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
83942025-08-26T21:53:51.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
83952025-08-26T21:53:51.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
83962025-08-26T21:53:51.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
83972025-08-26T21:53:51.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
83982025-08-26T21:53:51.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
83992025-08-26T21:53:51.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.53416ms iteration = 95 status = {"chicken_switches_updated":false}
84002025-08-26T21:53:51.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84012025-08-26T21:53:51.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
84022025-08-26T21:53:51.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
84032025-08-26T21:53:51.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
84042025-08-26T21:53:51.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
84052025-08-26T21:53:51.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
84062025-08-26T21:53:51.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84072025-08-26T21:53:51.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
84082025-08-26T21:53:51.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
84092025-08-26T21:53:51.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
84102025-08-26T21:53:51.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 48 reason = Timeout
84112025-08-26T21:53:51.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
84122025-08-26T21:53:51.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
84132025-08-26T21:53:51.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
84142025-08-26T21:53:51.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
84152025-08-26T21:53:51.689ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.268594ms iteration = 48 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
84162025-08-26T21:53:51.699ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
84172025-08-26T21:53:51.699ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.331511ms iteration = 48 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
84182025-08-26T21:53:55.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 380 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = f3066216-6567-4d69-b7fa-948598fa5799 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
84192025-08-26T21:53:56.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 96 reason = Timeout
84202025-08-26T21:53:56.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
84212025-08-26T21:53:56.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
84222025-08-26T21:53:56.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84232025-08-26T21:53:56.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
84242025-08-26T21:53:56.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
84252025-08-26T21:53:56.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
84262025-08-26T21:53:56.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
84272025-08-26T21:53:56.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.319632ms iteration = 96 status = {"chicken_switches_updated":false}
84282025-08-26T21:54:00.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = d4259fd1-5721-4392-a97d-5bfaabf51ab5 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
84292025-08-26T21:54:00.346ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = d4259fd1-5721-4392-a97d-5bfaabf51ab5 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
84302025-08-26T21:54:00.348ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2554 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = d4259fd1-5721-4392-a97d-5bfaabf51ab5 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
84312025-08-26T21:54:00.348ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 3647749e-f61f-4e81-9e19-7a90cd83a637 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
84322025-08-26T21:54:00.348ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 3647749e-f61f-4e81-9e19-7a90cd83a637 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
84332025-08-26T21:54:00.350ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 1837 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 3647749e-f61f-4e81-9e19-7a90cd83a637 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
84342025-08-26T21:54:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
84352025-08-26T21:54:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
84362025-08-26T21:54:01.664ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
84372025-08-26T21:54:01.664ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
84382025-08-26T21:54:01.664ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
84392025-08-26T21:54:01.664ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
84402025-08-26T21:54:01.664ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 17 reason = Timeout
84412025-08-26T21:54:01.664ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
84422025-08-26T21:54:01.664ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
84432025-08-26T21:54:01.664ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 9 reason = Timeout
84442025-08-26T21:54:01.664ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:44:01.664606166 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
84452025-08-26T21:54:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
84462025-08-26T21:54:01.664ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
84472025-08-26T21:54:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
84482025-08-26T21:54:01.664ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
84492025-08-26T21:54:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
84502025-08-26T21:54:01.664ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
84512025-08-26T21:54:01.664ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
84522025-08-26T21:54:01.664ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
84532025-08-26T21:54:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
84542025-08-26T21:54:01.665ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1345, tv_nsec: 523385221 } })
84552025-08-26T21:54:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
84562025-08-26T21:54:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
84572025-08-26T21:54:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
84582025-08-26T21:54:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
84592025-08-26T21:54:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
84602025-08-26T21:54:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
84612025-08-26T21:54:01.665ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1345, tv_nsec: 523569355 } })
84622025-08-26T21:54:01.665ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
84632025-08-26T21:54:01.665ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
84642025-08-26T21:54:01.665ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13\u{b5}s iteration = 10 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
84652025-08-26T21:54:01.665ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
84662025-08-26T21:54:01.665ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 883.86\u{b5}s iteration = 9 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
84672025-08-26T21:54:01.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
84682025-08-26T21:54:01.665ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
84692025-08-26T21:54:01.665ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.043683ms iteration = 9 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
84702025-08-26T21:54:01.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
84712025-08-26T21:54:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
84722025-08-26T21:54:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
84732025-08-26T21:54:01.666ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84742025-08-26T21:54:01.666ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:44:01.664606166 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
84752025-08-26T21:54:01.666ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 2.19866ms iteration = 9 status = {"expiration":"2025-08-26T21:44:01.664606166Z","pruned":[]}
84762025-08-26T21:54:01.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 97 reason = Timeout
84772025-08-26T21:54:01.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
84782025-08-26T21:54:01.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
84792025-08-26T21:54:01.667ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1345, tv_nsec: 525682503 } })
84802025-08-26T21:54:01.667ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84812025-08-26T21:54:01.667ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1345, tv_nsec: 525955139 } })
84822025-08-26T21:54:01.667ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
84832025-08-26T21:54:01.667ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84842025-08-26T21:54:01.667ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
84852025-08-26T21:54:01.667ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.360026ms iteration = 17 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
84862025-08-26T21:54:01.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84872025-08-26T21:54:01.669ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
84882025-08-26T21:54:01.669ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
84892025-08-26T21:54:01.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
84902025-08-26T21:54:01.671ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
84912025-08-26T21:54:01.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
84922025-08-26T21:54:01.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
84932025-08-26T21:54:01.672ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84942025-08-26T21:54:01.673ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
84952025-08-26T21:54:01.673ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
84962025-08-26T21:54:01.673ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
84972025-08-26T21:54:01.673ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
84982025-08-26T21:54:01.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
84992025-08-26T21:54:01.673ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 11 reason = Timeout
85002025-08-26T21:54:01.673ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
85012025-08-26T21:54:01.673ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 5.94\u{b5}s iteration = 11 status = "Disabled"
85022025-08-26T21:54:01.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
85032025-08-26T21:54:01.674ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
85042025-08-26T21:54:01.674ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
85052025-08-26T21:54:01.674ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
85062025-08-26T21:54:01.674ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
85072025-08-26T21:54:01.674ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 9 reason = Timeout
85082025-08-26T21:54:01.674ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
85092025-08-26T21:54:01.674ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
85102025-08-26T21:54:01.674ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 17 reason = Timeout
85112025-08-26T21:54:01.674ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 17 reason = Timeout
85122025-08-26T21:54:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
85132025-08-26T21:54:01.674ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
85142025-08-26T21:54:01.674ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
85152025-08-26T21:54:01.674ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 17 reason = Timeout
85162025-08-26T21:54:01.674ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
85172025-08-26T21:54:01.674ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 18 reason = Timeout
85182025-08-26T21:54:01.674ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
85192025-08-26T21:54:01.674ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
85202025-08-26T21:54:01.674ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
85212025-08-26T21:54:01.674ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 17 reason = Timeout
85222025-08-26T21:54:01.674ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
85232025-08-26T21:54:01.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
85242025-08-26T21:54:01.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
85252025-08-26T21:54:01.674ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85262025-08-26T21:54:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
85272025-08-26T21:54:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
85282025-08-26T21:54:01.674ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
85292025-08-26T21:54:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
85302025-08-26T21:54:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
85312025-08-26T21:54:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
85322025-08-26T21:54:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
85332025-08-26T21:54:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
85342025-08-26T21:54:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
85352025-08-26T21:54:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
85362025-08-26T21:54:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
85372025-08-26T21:54:01.676ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85382025-08-26T21:54:01.676ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85392025-08-26T21:54:01.676ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85402025-08-26T21:54:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
85412025-08-26T21:54:01.676ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
85422025-08-26T21:54:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
85432025-08-26T21:54:01.676ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85442025-08-26T21:54:01.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85452025-08-26T21:54:01.677ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
85462025-08-26T21:54:01.677ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 10.701981ms iteration = 97 status = {"chicken_switches_updated":false}
85472025-08-26T21:54:01.677ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85482025-08-26T21:54:01.677ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85492025-08-26T21:54:01.678ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
85502025-08-26T21:54:01.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
85512025-08-26T21:54:01.680ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
85522025-08-26T21:54:01.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
85532025-08-26T21:54:01.680ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85542025-08-26T21:54:01.682ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 7.596501ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
85552025-08-26T21:54:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
85562025-08-26T21:54:01.682ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
85572025-08-26T21:54:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
85582025-08-26T21:54:01.682ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
85592025-08-26T21:54:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
85602025-08-26T21:54:01.682ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
85612025-08-26T21:54:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
85622025-08-26T21:54:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
85632025-08-26T21:54:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
85642025-08-26T21:54:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
85652025-08-26T21:54:01.683ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
85662025-08-26T21:54:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
85672025-08-26T21:54:01.683ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
85682025-08-26T21:54:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
85692025-08-26T21:54:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
85702025-08-26T21:54:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
85712025-08-26T21:54:01.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
85722025-08-26T21:54:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
85732025-08-26T21:54:01.683ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
85742025-08-26T21:54:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
85752025-08-26T21:54:01.684ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
85762025-08-26T21:54:01.684ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
85772025-08-26T21:54:01.684ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 0 version = 2
85782025-08-26T21:54:01.684ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
85792025-08-26T21:54:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
85802025-08-26T21:54:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
85812025-08-26T21:54:01.684ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
85822025-08-26T21:54:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
85832025-08-26T21:54:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
85842025-08-26T21:54:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
85852025-08-26T21:54:01.684ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
85862025-08-26T21:54:01.684ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
85872025-08-26T21:54:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
85882025-08-26T21:54:01.685ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
85892025-08-26T21:54:01.685ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1345, tv_nsec: 543572086 } })
85902025-08-26T21:54:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
85912025-08-26T21:54:01.685ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
85922025-08-26T21:54:01.685ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85932025-08-26T21:54:01.685ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85942025-08-26T21:54:01.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
85952025-08-26T21:54:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
85962025-08-26T21:54:01.686ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 9 reason = Timeout
85972025-08-26T21:54:01.686ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 17 reason = Timeout
85982025-08-26T21:54:01.686ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
85992025-08-26T21:54:01.686ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
86002025-08-26T21:54:01.686ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 17 reason = Timeout
86012025-08-26T21:54:01.686ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
86022025-08-26T21:54:01.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 49 reason = Timeout
86032025-08-26T21:54:01.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
86042025-08-26T21:54:01.686ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
86052025-08-26T21:54:01.686ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86062025-08-26T21:54:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
86072025-08-26T21:54:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
86082025-08-26T21:54:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
86092025-08-26T21:54:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
86102025-08-26T21:54:01.687ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 9 reason = Timeout
86112025-08-26T21:54:01.687ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
86122025-08-26T21:54:01.687ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 17 reason = Timeout
86132025-08-26T21:54:01.687ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
86142025-08-26T21:54:01.687ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 9 reason = Timeout
86152025-08-26T21:54:01.687ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
86162025-08-26T21:54:01.687ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 9 reason = Timeout
86172025-08-26T21:54:01.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86182025-08-26T21:54:01.687ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
86192025-08-26T21:54:01.687ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 17 reason = Timeout
86202025-08-26T21:54:01.687ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 9 reason = Timeout
86212025-08-26T21:54:01.687ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
86222025-08-26T21:54:01.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 17 reason = Timeout
86232025-08-26T21:54:01.687ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 9.36\u{b5}s iteration = 17 status = {"disabled":true,"errors":[],"sps":[]}
86242025-08-26T21:54:01.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
86252025-08-26T21:54:01.688ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 17 reason = Timeout
86262025-08-26T21:54:01.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
86272025-08-26T21:54:01.688ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 17 reason = Timeout
86282025-08-26T21:54:01.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
86292025-08-26T21:54:01.688ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 17 reason = Timeout
86302025-08-26T21:54:01.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
86312025-08-26T21:54:01.688ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 17 reason = Timeout
86322025-08-26T21:54:01.688ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
86332025-08-26T21:54:01.688ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
86342025-08-26T21:54:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
86352025-08-26T21:54:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
86362025-08-26T21:54:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
86372025-08-26T21:54:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
86382025-08-26T21:54:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
86392025-08-26T21:54:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
86402025-08-26T21:54:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
86412025-08-26T21:54:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
86422025-08-26T21:54:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
86432025-08-26T21:54:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
86442025-08-26T21:54:01.689ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
86452025-08-26T21:54:01.689ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
86462025-08-26T21:54:01.689ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
86472025-08-26T21:54:01.689ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
86482025-08-26T21:54:01.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
86492025-08-26T21:54:01.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
86502025-08-26T21:54:01.690ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
86512025-08-26T21:54:01.690ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
86522025-08-26T21:54:01.690ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
86532025-08-26T21:54:01.690ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
86542025-08-26T21:54:01.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
86552025-08-26T21:54:01.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
86562025-08-26T21:54:01.691ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 3.478909ms iteration = 9 status = {"errors":[],"found_port_ok":[]}
86572025-08-26T21:54:01.691ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
86582025-08-26T21:54:01.691ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
86592025-08-26T21:54:01.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
86602025-08-26T21:54:01.692ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86612025-08-26T21:54:01.693ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 4.952012ms iteration = 17 status = {"errors":[],"garbage_collect_requested":[]}
86622025-08-26T21:54:01.693ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
86632025-08-26T21:54:01.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
86642025-08-26T21:54:01.694ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
86652025-08-26T21:54:01.694ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 6.815424ms iteration = 9 status = {"by_rx":{},"error":null}
86662025-08-26T21:54:01.695ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
86672025-08-26T21:54:01.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
86682025-08-26T21:54:01.696ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
86692025-08-26T21:54:01.696ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
86702025-08-26T21:54:01.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
86712025-08-26T21:54:01.697ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 4 version = 2
86722025-08-26T21:54:01.697ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
86732025-08-26T21:54:01.697ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-08-26 21:46:05.296205 UTC dns_group = external generation = 2
86742025-08-26T21:54:01.697ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 32.574175ms iteration = 9 status = {"generation":2}
86752025-08-26T21:54:01.697ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
86762025-08-26T21:54:01.697ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
86772025-08-26T21:54:01.698ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
86782025-08-26T21:54:01.698ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
86792025-08-26T21:54:01.698ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 23.602702ms iteration = 9 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
86802025-08-26T21:54:01.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
86812025-08-26T21:54:01.698ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 10.614349ms iteration = 17 status = {"errors":[],"finish_invoked_ok":[]}
86822025-08-26T21:54:01.701ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 13.509434ms iteration = 17 status = {"errors":[],"requests_created_ok":[]}
86832025-08-26T21:54:01.701ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
86842025-08-26T21:54:01.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
86852025-08-26T21:54:01.702ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 28.113154ms iteration = 18 status = {"physical_disks_added":0}
86862025-08-26T21:54:01.703ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
86872025-08-26T21:54:01.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
86882025-08-26T21:54:01.704ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
86892025-08-26T21:54:01.704ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86902025-08-26T21:54:01.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
86912025-08-26T21:54:01.707ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 19.596072ms iteration = 9 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
86922025-08-26T21:54:01.708ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
86932025-08-26T21:54:01.708ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
86942025-08-26T21:54:01.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
86952025-08-26T21:54:01.712ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 24.154645ms iteration = 17 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
86962025-08-26T21:54:01.729ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
86972025-08-26T21:54:01.729ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 55.231865ms iteration = 49 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
86982025-08-26T21:54:01.755ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
86992025-08-26T21:54:01.755ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
87002025-08-26T21:54:01.755ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
87012025-08-26T21:54:01.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
87022025-08-26T21:54:01.758ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87032025-08-26T21:54:01.758ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
87042025-08-26T21:54:01.758ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
87052025-08-26T21:54:01.758ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 70.795326ms iteration = 9 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
87062025-08-26T21:54:01.759ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
87072025-08-26T21:54:01.759ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
87082025-08-26T21:54:01.759ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
87092025-08-26T21:54:01.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
87102025-08-26T21:54:01.761ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
87112025-08-26T21:54:01.761ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 87.258497ms iteration = 17 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
87122025-08-26T21:54:01.762ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
87132025-08-26T21:54:01.762ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
87142025-08-26T21:54:01.762ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 76.572646ms iteration = 17 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
87152025-08-26T21:54:01.763ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
87162025-08-26T21:54:01.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
87172025-08-26T21:54:01.763ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
87182025-08-26T21:54:01.763ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
87192025-08-26T21:54:01.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
87202025-08-26T21:54:01.766ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 79.746678ms iteration = 49 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
87212025-08-26T21:54:01.766ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
87222025-08-26T21:54:01.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
87232025-08-26T21:54:01.766ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 79.005081ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
87242025-08-26T21:54:01.769ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
87252025-08-26T21:54:01.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
87262025-08-26T21:54:01.772ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
87272025-08-26T21:54:01.772ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
87282025-08-26T21:54:01.772ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-08-26 21:46:04.065217 UTC dns_group = internal generation = 1
87292025-08-26T21:54:01.772ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 108.031286ms iteration = 9 status = {"generation":1}
87302025-08-26T21:54:01.774ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87312025-08-26T21:54:01.775ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87322025-08-26T21:54:01.777ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
87332025-08-26T21:54:01.777ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
87342025-08-26T21:54:01.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
87352025-08-26T21:54:01.778ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
87362025-08-26T21:54:01.778ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
87372025-08-26T21:54:01.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
87382025-08-26T21:54:01.780ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 106.263776ms iteration = 17 status = 0
87392025-08-26T21:54:01.784ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
87402025-08-26T21:54:01.784ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 97.948648ms iteration = 17 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
87412025-08-26T21:54:01.786ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-08-26T21:45:59.866217Z, time_modified: 2025-08-26T21:45:59.866217Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }) }
87422025-08-26T21:54:01.786ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 121.790566ms iteration = 9 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]}},"default_endpoint":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
87432025-08-26T21:54:01.805ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
87442025-08-26T21:54:01.809ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
87452025-08-26T21:54:01.830ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
87462025-08-26T21:54:01.830ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
87472025-08-26T21:54:01.830ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "81aae32d-f440-4ca8-9308-083d4843ef9c", "content-length": "26", "date": "Tue, 26 Aug 2025 21:54:01 GMT"} })
87482025-08-26T21:54:01.830ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
87492025-08-26T21:54:01.831ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
87502025-08-26T21:54:01.840ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
87512025-08-26T21:54:01.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
87522025-08-26T21:54:01.844ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
87532025-08-26T21:54:01.844ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 157.662394ms iteration = 9 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
87542025-08-26T21:54:01.856ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
87552025-08-26T21:54:01.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
87562025-08-26T21:54:01.857ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87572025-08-26T21:54:01.859ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
87582025-08-26T21:54:01.859ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
87592025-08-26T21:54:01.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
87602025-08-26T21:54:01.861ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87612025-08-26T21:54:01.863ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
87622025-08-26T21:54:01.863ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
87632025-08-26T21:54:01.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
87642025-08-26T21:54:01.943ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
87652025-08-26T21:54:01.943ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
87662025-08-26T21:54:01.944ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb7c4cf2-12bf-4121-b831-6c63e12442cf", "content-length": "26", "date": "Tue, 26 Aug 2025 21:54:01 GMT"} })
87672025-08-26T21:54:01.944ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
87682025-08-26T21:54:01.945ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
87692025-08-26T21:54:01.959ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
87702025-08-26T21:54:01.959ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
87712025-08-26T21:54:01.959ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5705f21-1c9e-437a-962e-3db66ddb7419", "content-length": "26", "date": "Tue, 26 Aug 2025 21:54:01 GMT"} })
87722025-08-26T21:54:01.959ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
87732025-08-26T21:54:01.960ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
87742025-08-26T21:54:01.981ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
87752025-08-26T21:54:01.981ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
87762025-08-26T21:54:01.981ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "1474d9c0-61ac-4a4e-894c-c3cfff8ebdcb", "content-length": "372", "date": "Tue, 26 Aug 2025 21:54:01 GMT"} })
87772025-08-26T21:54:01.981ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
87782025-08-26T21:54:01.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
87792025-08-26T21:54:01.995ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
87802025-08-26T21:54:01.995ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
87812025-08-26T21:54:01.995ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "6ae44e7a-5bc8-4668-84ff-7ae4b52732d7", "date": "Tue, 26 Aug 2025 21:54:01 GMT"} })
87822025-08-26T21:54:01.995ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
87832025-08-26T21:54:01.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
87842025-08-26T21:54:01.997ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
87852025-08-26T21:54:01.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
87862025-08-26T21:54:02.000ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
87872025-08-26T21:54:02.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
87882025-08-26T21:54:02.003ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
87892025-08-26T21:54:02.003ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
87902025-08-26T21:54:02.003ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "48a1ab32-cf59-4075-bdde-50c8b59d2477", "date": "Tue, 26 Aug 2025 21:54:01 GMT"} })
87912025-08-26T21:54:02.004ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
87922025-08-26T21:54:02.004ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "bf5ee009-68d7-4cf5-9a07-b834f552bbd0", "date": "Tue, 26 Aug 2025 21:54:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
87932025-08-26T21:54:02.004ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
87942025-08-26T21:54:02.008ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 320.904174ms iteration = 17 status = {}
87952025-08-26T21:54:02.009ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "4583f495-8668-4e4d-92e2-7b83f954c8ce", "date": "Tue, 26 Aug 2025 21:54:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
87962025-08-26T21:54:02.010ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
87972025-08-26T21:54:02.012ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
87982025-08-26T21:54:02.012ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 347.554575ms iteration = 11 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
87992025-08-26T21:54:02.016ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
88002025-08-26T21:54:02.043ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
88012025-08-26T21:54:02.044ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "59f82a9e-0dc2-4e76-ae1f-6c2191603cac", "content-length": "691", "date": "Tue, 26 Aug 2025 21:54:02 GMT"} })
88022025-08-26T21:54:02.044ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
88032025-08-26T21:54:02.044ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
88042025-08-26T21:54:02.044ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
88052025-08-26T21:54:02.046ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 371.562135ms iteration = 17 status = {}
88062025-08-26T21:54:02.060ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
88072025-08-26T21:54:02.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
88082025-08-26T21:54:02.062ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 397.49406ms iteration = 17 status = null
88092025-08-26T21:54:02.075ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 411.172728ms iteration = 17 status = {}
88102025-08-26T21:54:02.083ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
88112025-08-26T21:54:02.083ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "ee673a6d-9a1f-40a3-b65c-bbdac8e79d1b", "date": "Tue, 26 Aug 2025 21:54:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
88122025-08-26T21:54:02.083ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
88132025-08-26T21:54:02.085ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "9c323305-0ebe-4c45-a4d0-fd68acb4c775", "date": "Tue, 26 Aug 2025 21:54:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
88142025-08-26T21:54:02.085ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
88152025-08-26T21:54:02.086ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "90e77382-68a2-41a8-83dc-8a4ca7305247", "date": "Tue, 26 Aug 2025 21:54:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
88162025-08-26T21:54:02.086ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
88172025-08-26T21:54:02.087ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
88182025-08-26T21:54:02.087ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 423.311142ms iteration = 11 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
88192025-08-26T21:54:02.121ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
88202025-08-26T21:54:02.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
88212025-08-26T21:54:02.124ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
88222025-08-26T21:54:02.124ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
88232025-08-26T21:54:02.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
88242025-08-26T21:54:02.133ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
88252025-08-26T21:54:02.133ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
88262025-08-26T21:54:02.133ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
88272025-08-26T21:54:02.133ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
88282025-08-26T21:54:02.133ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
88292025-08-26T21:54:02.133ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
88302025-08-26T21:54:02.133ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
88312025-08-26T21:54:02.133ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
88322025-08-26T21:54:02.133ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
88332025-08-26T21:54:02.133ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
88342025-08-26T21:54:02.134ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
88352025-08-26T21:54:02.134ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
88362025-08-26T21:54:02.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
88372025-08-26T21:54:02.136ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
88382025-08-26T21:54:02.136ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
88392025-08-26T21:54:02.245ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "80a5bb01-e4bf-4f68-9907-c8f4a003fac7", "date": "Tue, 26 Aug 2025 21:54:02 GMT"} })
88402025-08-26T21:54:02.245ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
88412025-08-26T21:54:02.245ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
88422025-08-26T21:54:02.245ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "c12da763-0aa1-4ade-9b30-a257cd4f5cd5", "content-length": "682", "date": "Tue, 26 Aug 2025 21:54:01 GMT"} })
88432025-08-26T21:54:02.245ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
88442025-08-26T21:54:02.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
88452025-08-26T21:54:02.248ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
88462025-08-26T21:54:02.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
88472025-08-26T21:54:02.252ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
88482025-08-26T21:54:02.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
88492025-08-26T21:54:02.254ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
88502025-08-26T21:54:02.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
88512025-08-26T21:54:02.261ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 586.714046ms iteration = 17 status = {}
88522025-08-26T21:54:05.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 333 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = b94a3e45-9837-4ff2-a8e7-5598e5f63569 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
88532025-08-26T21:54:06.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 98 reason = Timeout
88542025-08-26T21:54:06.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
88552025-08-26T21:54:06.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
88562025-08-26T21:54:06.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88572025-08-26T21:54:06.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
88582025-08-26T21:54:06.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
88592025-08-26T21:54:06.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
88602025-08-26T21:54:06.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
88612025-08-26T21:54:06.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.084332ms iteration = 98 status = {"chicken_switches_updated":false}
88622025-08-26T21:54:11.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 99 reason = Timeout
88632025-08-26T21:54:11.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
88642025-08-26T21:54:11.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
88652025-08-26T21:54:11.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88662025-08-26T21:54:11.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
88672025-08-26T21:54:11.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
88682025-08-26T21:54:11.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
88692025-08-26T21:54:11.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
88702025-08-26T21:54:11.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88712025-08-26T21:54:11.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88722025-08-26T21:54:11.675ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
88732025-08-26T21:54:11.675ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.133063ms iteration = 99 status = {"chicken_switches_updated":false}
88742025-08-26T21:54:11.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88752025-08-26T21:54:11.678ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
88762025-08-26T21:54:11.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88772025-08-26T21:54:11.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88782025-08-26T21:54:11.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88792025-08-26T21:54:11.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88802025-08-26T21:54:11.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88812025-08-26T21:54:11.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
88822025-08-26T21:54:11.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88832025-08-26T21:54:11.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
88842025-08-26T21:54:11.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 50 reason = Timeout
88852025-08-26T21:54:11.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
88862025-08-26T21:54:11.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
88872025-08-26T21:54:11.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
88882025-08-26T21:54:11.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
88892025-08-26T21:54:11.689ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.004818ms iteration = 50 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
88902025-08-26T21:54:11.699ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
88912025-08-26T21:54:11.699ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.513602ms iteration = 50 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
88922025-08-26T21:54:15.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = a497dd11-4fc5-47f6-9427-17bf36e9cf2c roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
88932025-08-26T21:54:15.345ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = a497dd11-4fc5-47f6-9427-17bf36e9cf2c resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
88942025-08-26T21:54:15.348ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3215 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = a497dd11-4fc5-47f6-9427-17bf36e9cf2c response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
88952025-08-26T21:54:15.348ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = eeaa71f3-4602-4244-989c-97607e3bb037 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
88962025-08-26T21:54:15.348ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = eeaa71f3-4602-4244-989c-97607e3bb037 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
88972025-08-26T21:54:15.351ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2483 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = eeaa71f3-4602-4244-989c-97607e3bb037 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
88982025-08-26T21:54:15.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 328 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = a933548c-47e0-4f69-b213-2ebae3a02a2b response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
88992025-08-26T21:54:16.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 100 reason = Timeout
89002025-08-26T21:54:16.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
89012025-08-26T21:54:16.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
89022025-08-26T21:54:16.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89032025-08-26T21:54:16.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
89042025-08-26T21:54:16.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
89052025-08-26T21:54:16.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
89062025-08-26T21:54:16.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
89072025-08-26T21:54:16.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.290875ms iteration = 100 status = {"chicken_switches_updated":false}
89082025-08-26T21:54:21.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 101 reason = Timeout
89092025-08-26T21:54:21.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
89102025-08-26T21:54:21.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
89112025-08-26T21:54:21.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89122025-08-26T21:54:21.670ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
89132025-08-26T21:54:21.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
89142025-08-26T21:54:21.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
89152025-08-26T21:54:21.672ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
89162025-08-26T21:54:21.672ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.294102ms iteration = 101 status = {"chicken_switches_updated":false}
89172025-08-26T21:54:21.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 51 reason = Timeout
89182025-08-26T21:54:21.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
89192025-08-26T21:54:21.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
89202025-08-26T21:54:21.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89212025-08-26T21:54:21.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
89222025-08-26T21:54:21.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
89232025-08-26T21:54:21.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
89242025-08-26T21:54:21.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
89252025-08-26T21:54:21.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
89262025-08-26T21:54:21.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89272025-08-26T21:54:21.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
89282025-08-26T21:54:21.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
89292025-08-26T21:54:21.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
89302025-08-26T21:54:21.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 51 reason = Timeout
89312025-08-26T21:54:21.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
89322025-08-26T21:54:21.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
89332025-08-26T21:54:21.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
89342025-08-26T21:54:21.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
89352025-08-26T21:54:21.689ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.058198ms iteration = 51 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
89362025-08-26T21:54:21.698ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
89372025-08-26T21:54:21.698ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.902849ms iteration = 51 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
89382025-08-26T21:54:25.353ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 631 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = b7cf30bc-df70-4a8d-ac7d-3aa732bc816e response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
89392025-08-26T21:54:26.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 102 reason = Timeout
89402025-08-26T21:54:26.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
89412025-08-26T21:54:26.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
89422025-08-26T21:54:26.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89432025-08-26T21:54:26.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
89442025-08-26T21:54:26.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
89452025-08-26T21:54:26.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
89462025-08-26T21:54:26.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
89472025-08-26T21:54:26.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.777235ms iteration = 102 status = {"chicken_switches_updated":false}
89482025-08-26T21:54:31.664ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
89492025-08-26T21:54:31.664ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 18 reason = Timeout
89502025-08-26T21:54:31.664ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
89512025-08-26T21:54:31.664ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
89522025-08-26T21:54:31.664ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
89532025-08-26T21:54:31.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
89542025-08-26T21:54:31.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
89552025-08-26T21:54:31.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 103 reason = Timeout
89562025-08-26T21:54:31.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
89572025-08-26T21:54:31.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
89582025-08-26T21:54:31.666ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1375, tv_nsec: 525414637 } })
89592025-08-26T21:54:31.666ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1375, tv_nsec: 525447488 } })
89602025-08-26T21:54:31.666ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.406264ms iteration = 18 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
89612025-08-26T21:54:31.667ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
89622025-08-26T21:54:31.667ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
89632025-08-26T21:54:31.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89642025-08-26T21:54:31.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
89652025-08-26T21:54:31.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
89662025-08-26T21:54:31.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
89672025-08-26T21:54:31.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
89682025-08-26T21:54:31.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.865804ms iteration = 103 status = {"chicken_switches_updated":false}
89692025-08-26T21:54:31.674ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 18 reason = Timeout
89702025-08-26T21:54:31.674ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
89712025-08-26T21:54:31.674ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
89722025-08-26T21:54:31.674ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 19 reason = Timeout
89732025-08-26T21:54:31.674ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
89742025-08-26T21:54:31.674ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
89752025-08-26T21:54:31.674ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 18 reason = Timeout
89762025-08-26T21:54:31.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 52 reason = Timeout
89772025-08-26T21:54:31.674ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
89782025-08-26T21:54:31.674ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 18 reason = Timeout
89792025-08-26T21:54:31.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
89802025-08-26T21:54:31.674ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 18 reason = Timeout
89812025-08-26T21:54:31.674ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
89822025-08-26T21:54:31.674ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
89832025-08-26T21:54:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
89842025-08-26T21:54:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
89852025-08-26T21:54:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
89862025-08-26T21:54:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
89872025-08-26T21:54:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
89882025-08-26T21:54:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
89892025-08-26T21:54:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
89902025-08-26T21:54:31.676ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89912025-08-26T21:54:31.676ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89922025-08-26T21:54:31.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89932025-08-26T21:54:31.676ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89942025-08-26T21:54:31.676ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89952025-08-26T21:54:31.678ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
89962025-08-26T21:54:31.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
89972025-08-26T21:54:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
89982025-08-26T21:54:31.680ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
89992025-08-26T21:54:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
90002025-08-26T21:54:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
90012025-08-26T21:54:31.680ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
90022025-08-26T21:54:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
90032025-08-26T21:54:31.680ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
90042025-08-26T21:54:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
90052025-08-26T21:54:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
90062025-08-26T21:54:31.681ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 6.6714ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
90072025-08-26T21:54:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
90082025-08-26T21:54:31.681ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
90092025-08-26T21:54:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
90102025-08-26T21:54:31.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
90112025-08-26T21:54:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
90122025-08-26T21:54:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
90132025-08-26T21:54:31.682ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
90142025-08-26T21:54:31.682ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
90152025-08-26T21:54:31.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
90162025-08-26T21:54:31.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
90172025-08-26T21:54:31.683ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1375, tv_nsec: 541805517 } })
90182025-08-26T21:54:31.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
90192025-08-26T21:54:31.683ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
90202025-08-26T21:54:31.683ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
90212025-08-26T21:54:31.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
90222025-08-26T21:54:31.683ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90232025-08-26T21:54:31.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
90242025-08-26T21:54:31.684ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90252025-08-26T21:54:31.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90262025-08-26T21:54:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
90272025-08-26T21:54:31.687ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
90282025-08-26T21:54:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
90292025-08-26T21:54:31.687ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 18 reason = Timeout
90302025-08-26T21:54:31.688ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 18 reason = Timeout
90312025-08-26T21:54:31.688ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
90322025-08-26T21:54:31.688ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
90332025-08-26T21:54:31.688ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
90342025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 18 reason = Timeout
90352025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
90362025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 18 reason = Timeout
90372025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
90382025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 52 reason = Timeout
90392025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
90402025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 18 reason = Timeout
90412025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
90422025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 18 reason = Timeout
90432025-08-26T21:54:31.689ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
90442025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
90452025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 18 reason = Timeout
90462025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
90472025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 18 reason = Timeout
90482025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 18 reason = Timeout
90492025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 17.55\u{b5}s iteration = 18 status = {"disabled":true,"errors":[],"sps":[]}
90502025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
90512025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
90522025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
90532025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
90542025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
90552025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
90562025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
90572025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
90582025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
90592025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
90602025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
90612025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
90622025-08-26T21:54:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
90632025-08-26T21:54:31.690ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
90642025-08-26T21:54:31.690ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
90652025-08-26T21:54:31.690ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
90662025-08-26T21:54:31.690ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 15.780176ms iteration = 18 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
90672025-08-26T21:54:31.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
90682025-08-26T21:54:31.691ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
90692025-08-26T21:54:31.691ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90702025-08-26T21:54:31.691ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90712025-08-26T21:54:31.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
90722025-08-26T21:54:31.692ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
90732025-08-26T21:54:31.692ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
90742025-08-26T21:54:31.692ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
90752025-08-26T21:54:31.692ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
90762025-08-26T21:54:31.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
90772025-08-26T21:54:31.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
90782025-08-26T21:54:31.694ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 6.418415ms iteration = 18 status = {"errors":[],"garbage_collect_requested":[]}
90792025-08-26T21:54:31.694ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 6.613889ms iteration = 18 status = {"errors":[],"finish_invoked_ok":[]}
90802025-08-26T21:54:31.694ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 20.214715ms iteration = 19 status = {"physical_disks_added":0}
90812025-08-26T21:54:31.694ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 6.692861ms iteration = 52 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
90822025-08-26T21:54:31.695ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
90832025-08-26T21:54:31.695ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
90842025-08-26T21:54:31.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
90852025-08-26T21:54:31.695ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
90862025-08-26T21:54:31.695ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
90872025-08-26T21:54:31.695ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
90882025-08-26T21:54:31.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
90892025-08-26T21:54:31.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
90902025-08-26T21:54:31.696ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
90912025-08-26T21:54:31.696ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
90922025-08-26T21:54:31.696ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
90932025-08-26T21:54:31.696ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 8.023851ms iteration = 18 status = {"errors":[],"requests_created_ok":[]}
90942025-08-26T21:54:31.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
90952025-08-26T21:54:31.697ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90962025-08-26T21:54:31.697ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 9.856993ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
90972025-08-26T21:54:31.699ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
90982025-08-26T21:54:31.699ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
90992025-08-26T21:54:31.699ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 11.577621ms iteration = 18 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
91002025-08-26T21:54:31.700ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
91012025-08-26T21:54:31.700ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
91022025-08-26T21:54:31.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
91032025-08-26T21:54:31.712ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
91042025-08-26T21:54:31.712ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 37.600028ms iteration = 52 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
91052025-08-26T21:54:31.732ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
91062025-08-26T21:54:31.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
91072025-08-26T21:54:31.734ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
91082025-08-26T21:54:31.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
91092025-08-26T21:54:31.737ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91102025-08-26T21:54:31.739ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
91112025-08-26T21:54:31.739ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
91122025-08-26T21:54:31.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
91132025-08-26T21:54:31.741ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 67.352318ms iteration = 18 status = 0
91142025-08-26T21:54:31.742ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
91152025-08-26T21:54:31.742ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
91162025-08-26T21:54:31.742ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d7b2181-649c-4bc1-8742-c851d9e8ba01", "content-length": "26", "date": "Tue, 26 Aug 2025 21:54:31 GMT"} })
91172025-08-26T21:54:31.742ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
91182025-08-26T21:54:31.743ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
91192025-08-26T21:54:31.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
91202025-08-26T21:54:31.744ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
91212025-08-26T21:54:31.744ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
91222025-08-26T21:54:31.744ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
91232025-08-26T21:54:31.744ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d500830-3dc3-4028-bb61-bc7565c89dcc", "content-length": "26", "date": "Tue, 26 Aug 2025 21:54:31 GMT"} })
91242025-08-26T21:54:31.744ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
91252025-08-26T21:54:31.745ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
91262025-08-26T21:54:31.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
91272025-08-26T21:54:31.746ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
91282025-08-26T21:54:31.747ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 59.943311ms iteration = 18 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
91292025-08-26T21:54:31.753ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
91302025-08-26T21:54:31.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
91312025-08-26T21:54:31.755ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91322025-08-26T21:54:31.757ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
91332025-08-26T21:54:31.757ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 69.130139ms iteration = 18 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
91342025-08-26T21:54:31.757ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
91352025-08-26T21:54:31.757ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
91362025-08-26T21:54:31.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
91372025-08-26T21:54:31.759ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91382025-08-26T21:54:31.761ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
91392025-08-26T21:54:31.761ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
91402025-08-26T21:54:31.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
91412025-08-26T21:54:31.801ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
91422025-08-26T21:54:31.801ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
91432025-08-26T21:54:31.801ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e50aa33b-3c07-4a22-82c4-0528a9953538", "content-length": "26", "date": "Tue, 26 Aug 2025 21:54:31 GMT"} })
91442025-08-26T21:54:31.801ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
91452025-08-26T21:54:31.803ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
91462025-08-26T21:54:31.874ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 210.281941ms iteration = 18 status = {}
91472025-08-26T21:54:31.886ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
91482025-08-26T21:54:31.886ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
91492025-08-26T21:54:31.886ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "588beb5f-876c-4d5f-8070-ad5340afa269", "content-length": "372", "date": "Tue, 26 Aug 2025 21:54:31 GMT"} })
91502025-08-26T21:54:31.886ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
91512025-08-26T21:54:31.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
91522025-08-26T21:54:31.934ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
91532025-08-26T21:54:31.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
91542025-08-26T21:54:31.936ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 272.166286ms iteration = 18 status = null
91552025-08-26T21:54:31.941ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
91562025-08-26T21:54:31.941ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
91572025-08-26T21:54:31.942ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "209e5f46-bfed-4197-94aa-586b09957c35", "date": "Tue, 26 Aug 2025 21:54:31 GMT"} })
91582025-08-26T21:54:31.942ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
91592025-08-26T21:54:31.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
91602025-08-26T21:54:31.944ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
91612025-08-26T21:54:31.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
91622025-08-26T21:54:31.947ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
91632025-08-26T21:54:31.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
91642025-08-26T21:54:31.950ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
91652025-08-26T21:54:31.950ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
91662025-08-26T21:54:31.951ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "364b535a-687c-4c85-a256-634ea10088ed", "date": "Tue, 26 Aug 2025 21:54:31 GMT"} })
91672025-08-26T21:54:31.953ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 264.973153ms iteration = 18 status = {}
91682025-08-26T21:54:31.961ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
91692025-08-26T21:54:31.961ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3e3a1d4-7005-4074-a069-7d06bd5979cd", "content-length": "691", "date": "Tue, 26 Aug 2025 21:54:31 GMT"} })
91702025-08-26T21:54:31.961ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
91712025-08-26T21:54:31.961ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
91722025-08-26T21:54:31.961ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
91732025-08-26T21:54:31.963ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 288.849661ms iteration = 18 status = {}
91742025-08-26T21:54:32.029ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
91752025-08-26T21:54:32.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
91762025-08-26T21:54:32.032ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
91772025-08-26T21:54:32.032ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
91782025-08-26T21:54:32.032ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
91792025-08-26T21:54:32.045ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
91802025-08-26T21:54:32.045ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
91812025-08-26T21:54:32.045ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
91822025-08-26T21:54:32.045ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
91832025-08-26T21:54:32.046ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
91842025-08-26T21:54:32.046ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
91852025-08-26T21:54:32.046ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
91862025-08-26T21:54:32.046ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
91872025-08-26T21:54:32.046ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
91882025-08-26T21:54:32.046ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
91892025-08-26T21:54:32.046ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
91902025-08-26T21:54:32.046ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
91912025-08-26T21:54:32.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
91922025-08-26T21:54:32.048ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
91932025-08-26T21:54:32.048ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
91942025-08-26T21:54:32.161ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "a9b3bf01-1e35-4d46-9e82-ac28791bed04", "date": "Tue, 26 Aug 2025 21:54:31 GMT"} })
91952025-08-26T21:54:32.161ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
91962025-08-26T21:54:32.161ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
91972025-08-26T21:54:32.162ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7914ce7-d28b-412b-a339-46b9083aac39", "content-length": "682", "date": "Tue, 26 Aug 2025 21:54:31 GMT"} })
91982025-08-26T21:54:32.162ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
91992025-08-26T21:54:32.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
92002025-08-26T21:54:32.166ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
92012025-08-26T21:54:32.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
92022025-08-26T21:54:32.170ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
92032025-08-26T21:54:32.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
92042025-08-26T21:54:32.172ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
92052025-08-26T21:54:32.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
92062025-08-26T21:54:32.180ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 505.636108ms iteration = 18 status = {}
92072025-08-26T21:54:32.406ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 3
92082025-08-26T21:54:32.406ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221
92092025-08-26T21:54:32.437ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
92102025-08-26T21:54:32.437ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::a]:56495
92112025-08-26T21:54:32.437ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:56495 req_id = ebb86194-5314-4b9b-8d4b-8aa152978a9c roles = RoleSet { roles: {} } uri = /metrics/producers
92122025-08-26T21:54:32.437ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:56495 req_id = ebb86194-5314-4b9b-8d4b-8aa152978a9c resource = Database result = Ok(()) uri = /metrics/producers
92132025-08-26T21:54:32.481ZINFOnexus: assigned collector to new producer collector_id = f843ff0e-eef8-49c4-b428-23e15df1def3 file = nexus/src/app/oximeter.rs:76 producer_id = 4996dcc4-c530-4f64-b407-fa372462bd9e
92142025-08-26T21:54:32.481ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 44104 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:56495 req_id = ebb86194-5314-4b9b-8d4b-8aa152978a9c response_code = 201 uri = /metrics/producers
92152025-08-26T21:54:32.481ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "ebb86194-5314-4b9b-8d4b-8aa152978a9c", "content-length": "41", "date": "Tue, 26 Aug 2025 21:54:32 GMT"} })
92162025-08-26T21:54:32.483ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
92172025-08-26T21:54:35.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 349 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 9e0a71a3-a2bf-4d10-aa85-0da8a467963f response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
92182025-08-26T21:54:36.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 104 reason = Timeout
92192025-08-26T21:54:36.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
92202025-08-26T21:54:36.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
92212025-08-26T21:54:36.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92222025-08-26T21:54:36.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
92232025-08-26T21:54:36.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
92242025-08-26T21:54:36.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
92252025-08-26T21:54:36.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
92262025-08-26T21:54:36.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.160684ms iteration = 104 status = {"chicken_switches_updated":false}
92272025-08-26T21:54:41.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 105 reason = Timeout
92282025-08-26T21:54:41.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
92292025-08-26T21:54:41.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
92302025-08-26T21:54:41.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92312025-08-26T21:54:41.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
92322025-08-26T21:54:41.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
92332025-08-26T21:54:41.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
92342025-08-26T21:54:41.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
92352025-08-26T21:54:41.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 53 reason = Timeout
92362025-08-26T21:54:41.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.102433ms iteration = 105 status = {"chicken_switches_updated":false}
92372025-08-26T21:54:41.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
92382025-08-26T21:54:41.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
92392025-08-26T21:54:41.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92402025-08-26T21:54:41.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
92412025-08-26T21:54:41.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
92422025-08-26T21:54:41.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
92432025-08-26T21:54:41.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
92442025-08-26T21:54:41.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
92452025-08-26T21:54:41.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92462025-08-26T21:54:41.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
92472025-08-26T21:54:41.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
92482025-08-26T21:54:41.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
92492025-08-26T21:54:41.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 53 reason = Timeout
92502025-08-26T21:54:41.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
92512025-08-26T21:54:41.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
92522025-08-26T21:54:41.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
92532025-08-26T21:54:41.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
92542025-08-26T21:54:41.691ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.278106ms iteration = 53 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
92552025-08-26T21:54:41.699ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
92562025-08-26T21:54:41.699ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.148337ms iteration = 53 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
92572025-08-26T21:54:45.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 2a1c9d9b-4fe9-494b-a5d8-adf5d46f7777 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
92582025-08-26T21:54:45.345ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 2a1c9d9b-4fe9-494b-a5d8-adf5d46f7777 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
92592025-08-26T21:54:45.347ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2924 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 2a1c9d9b-4fe9-494b-a5d8-adf5d46f7777 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
92602025-08-26T21:54:45.348ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = f8216565-96fa-4372-bfb7-76a03482b2f9 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
92612025-08-26T21:54:45.348ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = f8216565-96fa-4372-bfb7-76a03482b2f9 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
92622025-08-26T21:54:45.350ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2469 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = f8216565-96fa-4372-bfb7-76a03482b2f9 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
92632025-08-26T21:54:45.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 359 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = e051475d-98a7-4028-ba8a-de89d348a1a9 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
92642025-08-26T21:54:46.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 106 reason = Timeout
92652025-08-26T21:54:46.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
92662025-08-26T21:54:46.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
92672025-08-26T21:54:46.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92682025-08-26T21:54:46.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
92692025-08-26T21:54:46.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
92702025-08-26T21:54:46.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
92712025-08-26T21:54:46.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
92722025-08-26T21:54:46.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.708994ms iteration = 106 status = {"chicken_switches_updated":false}
92732025-08-26T21:54:51.667ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 107 reason = Timeout
92742025-08-26T21:54:51.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
92752025-08-26T21:54:51.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
92762025-08-26T21:54:51.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92772025-08-26T21:54:51.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
92782025-08-26T21:54:51.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
92792025-08-26T21:54:51.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
92802025-08-26T21:54:51.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
92812025-08-26T21:54:51.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.704161ms iteration = 107 status = {"chicken_switches_updated":false}
92822025-08-26T21:54:51.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 54 reason = Timeout
92832025-08-26T21:54:51.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
92842025-08-26T21:54:51.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
92852025-08-26T21:54:51.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92862025-08-26T21:54:51.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
92872025-08-26T21:54:51.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
92882025-08-26T21:54:51.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
92892025-08-26T21:54:51.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
92902025-08-26T21:54:51.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
92912025-08-26T21:54:51.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92922025-08-26T21:54:51.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 54 reason = Timeout
92932025-08-26T21:54:51.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
92942025-08-26T21:54:51.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
92952025-08-26T21:54:51.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
92962025-08-26T21:54:51.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
92972025-08-26T21:54:51.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
92982025-08-26T21:54:51.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
92992025-08-26T21:54:51.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
93002025-08-26T21:54:51.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.936289ms iteration = 54 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
93012025-08-26T21:54:51.700ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
93022025-08-26T21:54:51.700ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.785744ms iteration = 54 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
93032025-08-26T21:54:55.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 594 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 185abd2a-6fcf-4a07-af64-ecd824bc2ecd response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
93042025-08-26T21:54:56.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 108 reason = Timeout
93052025-08-26T21:54:56.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
93062025-08-26T21:54:56.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
93072025-08-26T21:54:56.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93082025-08-26T21:54:56.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
93092025-08-26T21:54:56.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
93102025-08-26T21:54:56.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
93112025-08-26T21:54:56.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
93122025-08-26T21:54:56.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.300977ms iteration = 108 status = {"chicken_switches_updated":false}
93132025-08-26T21:55:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
93142025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
93152025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 10 reason = Timeout
93162025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
93172025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
93182025-08-26T21:55:01.665ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:45:01.664987219 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
93192025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
93202025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
93212025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
93222025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
93232025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
93242025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
93252025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 19 reason = Timeout
93262025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
93272025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
93282025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 19 reason = Timeout
93292025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
93302025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
93312025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 19 reason = Timeout
93322025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
93332025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 11 reason = Timeout
93342025-08-26T21:55:01.665ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
93352025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 14.72\u{b5}s iteration = 11 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
93362025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
93372025-08-26T21:55:01.665ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1405, tv_nsec: 524069127 } })
93382025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
93392025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
93402025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
93412025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
93422025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
93432025-08-26T21:55:01.665ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1405, tv_nsec: 524251751 } })
93442025-08-26T21:55:01.665ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
93452025-08-26T21:55:01.666ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.003833ms iteration = 10 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
93462025-08-26T21:55:01.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 109 reason = Timeout
93472025-08-26T21:55:01.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
93482025-08-26T21:55:01.666ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
93492025-08-26T21:55:01.666ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.227798ms iteration = 10 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
93502025-08-26T21:55:01.666ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
93512025-08-26T21:55:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
93522025-08-26T21:55:01.666ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
93532025-08-26T21:55:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
93542025-08-26T21:55:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
93552025-08-26T21:55:01.667ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93562025-08-26T21:55:01.667ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:45:01.664987219 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
93572025-08-26T21:55:01.667ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 2.938956ms iteration = 10 status = {"expiration":"2025-08-26T21:45:01.664987219Z","pruned":[]}
93582025-08-26T21:55:01.667ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1405, tv_nsec: 526510612 } })
93592025-08-26T21:55:01.667ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1405, tv_nsec: 526565453 } })
93602025-08-26T21:55:01.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93612025-08-26T21:55:01.668ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
93622025-08-26T21:55:01.668ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
93632025-08-26T21:55:01.668ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.334526ms iteration = 19 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
93642025-08-26T21:55:01.668ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93652025-08-26T21:55:01.668ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93662025-08-26T21:55:01.671ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
93672025-08-26T21:55:01.671ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
93682025-08-26T21:55:01.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
93692025-08-26T21:55:01.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
93702025-08-26T21:55:01.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
93712025-08-26T21:55:01.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
93722025-08-26T21:55:01.673ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 12 reason = Timeout
93732025-08-26T21:55:01.673ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
93742025-08-26T21:55:01.673ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 10.24\u{b5}s iteration = 12 status = "Disabled"
93752025-08-26T21:55:01.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
93762025-08-26T21:55:01.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
93772025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.97689ms iteration = 109 status = {"chicken_switches_updated":false}
93782025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
93792025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
93802025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
93812025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
93822025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
93832025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 20 reason = Timeout
93842025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 19 reason = Timeout
93852025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
93862025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 19 reason = Timeout
93872025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
93882025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
93892025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 10 reason = Timeout
93902025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 5 reason = Timeout
93912025-08-26T21:55:01.674ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
93922025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
93932025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 19 reason = Timeout
93942025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
93952025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
93962025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 19 reason = Timeout
93972025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
93982025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 19 reason = Timeout
93992025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
94002025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 55 reason = Timeout
94012025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
94022025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
94032025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
94042025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
94052025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
94062025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
94072025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
94082025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
94092025-08-26T21:55:01.674ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
94102025-08-26T21:55:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
94112025-08-26T21:55:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
94122025-08-26T21:55:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
94132025-08-26T21:55:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
94142025-08-26T21:55:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
94152025-08-26T21:55:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
94162025-08-26T21:55:01.675ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
94172025-08-26T21:55:01.675ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
94182025-08-26T21:55:01.675ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
94192025-08-26T21:55:01.675ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
94202025-08-26T21:55:01.675ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
94212025-08-26T21:55:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
94222025-08-26T21:55:01.675ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
94232025-08-26T21:55:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
94242025-08-26T21:55:01.675ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
94252025-08-26T21:55:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
94262025-08-26T21:55:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
94272025-08-26T21:55:01.676ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94282025-08-26T21:55:01.677ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94292025-08-26T21:55:01.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94302025-08-26T21:55:01.678ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94312025-08-26T21:55:01.678ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94322025-08-26T21:55:01.678ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94332025-08-26T21:55:01.678ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94342025-08-26T21:55:01.678ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94352025-08-26T21:55:01.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
94362025-08-26T21:55:01.680ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
94372025-08-26T21:55:01.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
94382025-08-26T21:55:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
94392025-08-26T21:55:01.682ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
94402025-08-26T21:55:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
94412025-08-26T21:55:01.683ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
94422025-08-26T21:55:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
94432025-08-26T21:55:01.683ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
94442025-08-26T21:55:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
94452025-08-26T21:55:01.683ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
94462025-08-26T21:55:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
94472025-08-26T21:55:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
94482025-08-26T21:55:01.683ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
94492025-08-26T21:55:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
94502025-08-26T21:55:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
94512025-08-26T21:55:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
94522025-08-26T21:55:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
94532025-08-26T21:55:01.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
94542025-08-26T21:55:01.684ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94552025-08-26T21:55:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
94562025-08-26T21:55:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
94572025-08-26T21:55:01.684ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
94582025-08-26T21:55:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
94592025-08-26T21:55:01.685ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
94602025-08-26T21:55:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
94612025-08-26T21:55:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
94622025-08-26T21:55:01.685ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94632025-08-26T21:55:01.686ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
94642025-08-26T21:55:01.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
94652025-08-26T21:55:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
94662025-08-26T21:55:01.686ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 10 reason = Timeout
94672025-08-26T21:55:01.686ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
94682025-08-26T21:55:01.686ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 19 reason = Timeout
94692025-08-26T21:55:01.686ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
94702025-08-26T21:55:01.686ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 19 reason = Timeout
94712025-08-26T21:55:01.686ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
94722025-08-26T21:55:01.686ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
94732025-08-26T21:55:01.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 55 reason = Timeout
94742025-08-26T21:55:01.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
94752025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
94762025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
94772025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 10 reason = Timeout
94782025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
94792025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 19 reason = Timeout
94802025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
94812025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
94822025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
94832025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 19 reason = Timeout
94842025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
94852025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
94862025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
94872025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 10 reason = Timeout
94882025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
94892025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 19 reason = Timeout
94902025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
94912025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
94922025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 19 reason = Timeout
94932025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
94942025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
94952025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 10 reason = Timeout
94962025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
94972025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 19 reason = Timeout
94982025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
94992025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 19 reason = Timeout
95002025-08-26T21:55:01.687ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
95012025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
95022025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 10 reason = Timeout
95032025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 19 reason = Timeout
95042025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
95052025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 7.7\u{b5}s iteration = 19 status = {"disabled":true,"errors":[],"sps":[]}
95062025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
95072025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
95082025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
95092025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
95102025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
95112025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
95122025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
95132025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
95142025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
95152025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
95162025-08-26T21:55:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
95172025-08-26T21:55:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
95182025-08-26T21:55:01.689ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95192025-08-26T21:55:01.689ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
95202025-08-26T21:55:01.689ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
95212025-08-26T21:55:01.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
95222025-08-26T21:55:01.690ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
95232025-08-26T21:55:01.690ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
95242025-08-26T21:55:01.690ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
95252025-08-26T21:55:01.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
95262025-08-26T21:55:01.691ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 4.615604ms iteration = 19 status = {"errors":[],"garbage_collect_requested":[]}
95272025-08-26T21:55:01.695ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
95282025-08-26T21:55:01.695ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
95292025-08-26T21:55:01.695ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
95302025-08-26T21:55:01.695ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
95312025-08-26T21:55:01.695ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95322025-08-26T21:55:01.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
95332025-08-26T21:55:01.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
95342025-08-26T21:55:01.704ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
95352025-08-26T21:55:01.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
95362025-08-26T21:55:01.707ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 20.37203ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
95372025-08-26T21:55:01.713ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 26.395755ms iteration = 19 status = {"errors":[],"finish_invoked_ok":[]}
95382025-08-26T21:55:01.719ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95392025-08-26T21:55:01.722ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
95402025-08-26T21:55:01.722ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
95412025-08-26T21:55:01.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
95422025-08-26T21:55:01.725ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 38.066378ms iteration = 10 status = {"errors":[],"found_port_ok":[]}
95432025-08-26T21:55:01.729ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
95442025-08-26T21:55:01.729ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
95452025-08-26T21:55:01.729ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 42.463818ms iteration = 10 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
95462025-08-26T21:55:01.732ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
95472025-08-26T21:55:01.732ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 44.998385ms iteration = 10 status = {"by_rx":{},"error":null}
95482025-08-26T21:55:01.732ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 45.434444ms iteration = 19 status = {"errors":[],"requests_created_ok":[]}
95492025-08-26T21:55:01.739ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
95502025-08-26T21:55:01.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
95512025-08-26T21:55:01.760ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95522025-08-26T21:55:01.763ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95532025-08-26T21:55:01.765ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
95542025-08-26T21:55:01.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
95552025-08-26T21:55:01.766ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95562025-08-26T21:55:01.766ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
95572025-08-26T21:55:01.766ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
95582025-08-26T21:55:01.766ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95592025-08-26T21:55:01.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
95602025-08-26T21:55:01.767ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95612025-08-26T21:55:01.767ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
95622025-08-26T21:55:01.767ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
95632025-08-26T21:55:01.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
95642025-08-26T21:55:01.769ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 81.855846ms iteration = 10 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
95652025-08-26T21:55:01.770ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
95662025-08-26T21:55:01.770ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
95672025-08-26T21:55:01.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
95682025-08-26T21:55:01.771ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
95692025-08-26T21:55:01.771ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
95702025-08-26T21:55:01.771ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1405, tv_nsec: 630100117 } })
95712025-08-26T21:55:01.771ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
95722025-08-26T21:55:01.771ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
95732025-08-26T21:55:01.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
95742025-08-26T21:55:01.772ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
95752025-08-26T21:55:01.772ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
95762025-08-26T21:55:01.772ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
95772025-08-26T21:55:01.772ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac374440-1feb-49d4-abc4-c84bc893d20f", "content-length": "26", "date": "Tue, 26 Aug 2025 21:55:01 GMT"} })
95782025-08-26T21:55:01.772ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
95792025-08-26T21:55:01.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
95802025-08-26T21:55:01.773ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95812025-08-26T21:55:01.773ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95822025-08-26T21:55:01.775ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
95832025-08-26T21:55:01.777ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
95842025-08-26T21:55:01.777ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
95852025-08-26T21:55:01.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
95862025-08-26T21:55:01.778ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
95872025-08-26T21:55:01.778ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
95882025-08-26T21:55:01.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
95892025-08-26T21:55:01.778ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
95902025-08-26T21:55:01.778ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
95912025-08-26T21:55:01.778ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 104.195388ms iteration = 10 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
95922025-08-26T21:55:01.790ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
95932025-08-26T21:55:01.790ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
95942025-08-26T21:55:01.790ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
95952025-08-26T21:55:01.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
95962025-08-26T21:55:01.794ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
95972025-08-26T21:55:01.794ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 119.809871ms iteration = 55 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
95982025-08-26T21:55:01.795ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95992025-08-26T21:55:01.795ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96002025-08-26T21:55:01.800ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
96012025-08-26T21:55:01.800ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
96022025-08-26T21:55:01.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
96032025-08-26T21:55:01.801ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
96042025-08-26T21:55:01.801ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
96052025-08-26T21:55:01.801ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
96062025-08-26T21:55:01.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
96072025-08-26T21:55:01.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
96082025-08-26T21:55:01.803ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 128.880815ms iteration = 5 status = {"nsuccess":5}
96092025-08-26T21:55:01.805ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96102025-08-26T21:55:01.806ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
96112025-08-26T21:55:01.806ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
96122025-08-26T21:55:01.806ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-08-26 21:46:04.065217 UTC dns_group = internal generation = 1
96132025-08-26T21:55:01.806ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 141.642093ms iteration = 10 status = {"generation":1}
96142025-08-26T21:55:01.808ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
96152025-08-26T21:55:01.808ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
96162025-08-26T21:55:01.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
96172025-08-26T21:55:01.809ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
96182025-08-26T21:55:01.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
96192025-08-26T21:55:01.812ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 125.916268ms iteration = 55 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
96202025-08-26T21:55:01.813ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
96212025-08-26T21:55:01.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
96222025-08-26T21:55:01.813ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
96232025-08-26T21:55:01.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
96242025-08-26T21:55:01.816ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
96252025-08-26T21:55:01.816ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
96262025-08-26T21:55:01.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
96272025-08-26T21:55:01.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
96282025-08-26T21:55:01.822ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 148.526238ms iteration = 20 status = {"physical_disks_added":0}
96292025-08-26T21:55:01.843ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 168.705633ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
96302025-08-26T21:55:01.847ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
96312025-08-26T21:55:01.847ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
96322025-08-26T21:55:01.847ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
96332025-08-26T21:55:01.847ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e6b8318-5149-4ee0-8f72-202bfbe398d7", "content-length": "26", "date": "Tue, 26 Aug 2025 21:55:01 GMT"} })
96342025-08-26T21:55:01.847ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
96352025-08-26T21:55:01.849ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
96362025-08-26T21:55:01.849ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 175.132708ms iteration = 19 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
96372025-08-26T21:55:01.849ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
96382025-08-26T21:55:01.852ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
96392025-08-26T21:55:01.852ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 0 version = 2
96402025-08-26T21:55:01.852ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
96412025-08-26T21:55:01.852ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
96422025-08-26T21:55:01.852ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
96432025-08-26T21:55:01.852ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 165.511811ms iteration = 19 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
96442025-08-26T21:55:01.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
96452025-08-26T21:55:01.852ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
96462025-08-26T21:55:01.855ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96472025-08-26T21:55:01.860ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
96482025-08-26T21:55:01.860ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
96492025-08-26T21:55:01.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
96502025-08-26T21:55:01.863ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 4 version = 2
96512025-08-26T21:55:01.863ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
96522025-08-26T21:55:01.863ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-08-26 21:46:05.296205 UTC dns_group = external generation = 2
96532025-08-26T21:55:01.863ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 198.122197ms iteration = 10 status = {"generation":2}
96542025-08-26T21:55:01.876ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
96552025-08-26T21:55:01.876ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 189.927591ms iteration = 19 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
96562025-08-26T21:55:01.877ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 190.516784ms iteration = 19 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
96572025-08-26T21:55:01.882ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96582025-08-26T21:55:01.885ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-08-26T21:45:59.866217Z, time_modified: 2025-08-26T21:45:59.866217Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }) }
96592025-08-26T21:55:01.885ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 220.673455ms iteration = 10 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]}},"default_endpoint":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
96602025-08-26T21:55:01.886ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
96612025-08-26T21:55:01.886ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
96622025-08-26T21:55:01.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
96632025-08-26T21:55:01.888ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 214.491195ms iteration = 19 status = 0
96642025-08-26T21:55:01.908ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
96652025-08-26T21:55:01.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
96662025-08-26T21:55:01.924ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
96672025-08-26T21:55:01.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
96682025-08-26T21:55:01.949ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
96692025-08-26T21:55:01.949ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
96702025-08-26T21:55:01.950ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3635c6c-cf3d-4b8b-bac0-cacc67a372d0", "content-length": "26", "date": "Tue, 26 Aug 2025 21:55:01 GMT"} })
96712025-08-26T21:55:01.950ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
96722025-08-26T21:55:01.952ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
96732025-08-26T21:55:01.953ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 288.042173ms iteration = 19 status = {}
96742025-08-26T21:55:01.985ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
96752025-08-26T21:55:01.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
96762025-08-26T21:55:01.987ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96772025-08-26T21:55:01.989ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
96782025-08-26T21:55:01.989ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
96792025-08-26T21:55:01.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
96802025-08-26T21:55:01.992ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96812025-08-26T21:55:01.994ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
96822025-08-26T21:55:01.994ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
96832025-08-26T21:55:01.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
96842025-08-26T21:55:02.008ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
96852025-08-26T21:55:02.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
96862025-08-26T21:55:02.011ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 346.154773ms iteration = 19 status = null
96872025-08-26T21:55:02.011ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
96882025-08-26T21:55:02.019ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
96892025-08-26T21:55:02.019ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "3875745c-c731-4d32-bc8a-43f255027e67", "date": "Tue, 26 Aug 2025 21:55:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
96902025-08-26T21:55:02.019ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
96912025-08-26T21:55:02.021ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "46aff450-61c0-4ddf-b09c-03a0505f86d7", "date": "Tue, 26 Aug 2025 21:55:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
96922025-08-26T21:55:02.021ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
96932025-08-26T21:55:02.023ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
96942025-08-26T21:55:02.023ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 358.596773ms iteration = 12 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
96952025-08-26T21:55:02.058ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
96962025-08-26T21:55:02.058ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
96972025-08-26T21:55:02.059ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa8e10fc-6170-4219-afc0-7e689845d05e", "content-length": "372", "date": "Tue, 26 Aug 2025 21:55:01 GMT"} })
96982025-08-26T21:55:02.059ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
96992025-08-26T21:55:02.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
97002025-08-26T21:55:02.077ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
97012025-08-26T21:55:02.077ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
97022025-08-26T21:55:02.078ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "c23b5a99-8e81-46ed-bca6-254cc248ad46", "date": "Tue, 26 Aug 2025 21:55:02 GMT"} })
97032025-08-26T21:55:02.078ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
97042025-08-26T21:55:02.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
97052025-08-26T21:55:02.081ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
97062025-08-26T21:55:02.081ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 394.878502ms iteration = 10 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
97072025-08-26T21:55:02.081ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
97082025-08-26T21:55:02.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
97092025-08-26T21:55:02.086ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
97102025-08-26T21:55:02.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
97112025-08-26T21:55:02.089ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
97122025-08-26T21:55:02.089ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
97132025-08-26T21:55:02.090ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "2d1e38ac-9107-49ba-b952-da0b2fc5fbfd", "date": "Tue, 26 Aug 2025 21:55:01 GMT"} })
97142025-08-26T21:55:02.092ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 404.889287ms iteration = 19 status = {}
97152025-08-26T21:55:02.101ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
97162025-08-26T21:55:02.102ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "addb1ac8-afcd-4219-a2db-89952e6fe31c", "date": "Tue, 26 Aug 2025 21:55:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
97172025-08-26T21:55:02.102ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
97182025-08-26T21:55:02.105ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "967abea1-5baf-4af2-8686-aecdf6486a02", "date": "Tue, 26 Aug 2025 21:55:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
97192025-08-26T21:55:02.105ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
97202025-08-26T21:55:02.107ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "fccae511-0d8d-43da-a559-69cf74e5bb71", "date": "Tue, 26 Aug 2025 21:55:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
97212025-08-26T21:55:02.107ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
97222025-08-26T21:55:02.110ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
97232025-08-26T21:55:02.110ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 445.567654ms iteration = 12 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
97242025-08-26T21:55:02.173ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
97252025-08-26T21:55:02.174ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "86d3b53f-e572-4397-a26e-a1a20c4f3006", "content-length": "691", "date": "Tue, 26 Aug 2025 21:55:02 GMT"} })
97262025-08-26T21:55:02.174ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
97272025-08-26T21:55:02.174ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
97282025-08-26T21:55:02.174ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
97292025-08-26T21:55:02.175ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 501.191408ms iteration = 19 status = {}
97302025-08-26T21:55:02.188ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
97312025-08-26T21:55:02.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
97322025-08-26T21:55:02.191ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
97332025-08-26T21:55:02.191ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
97342025-08-26T21:55:02.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
97352025-08-26T21:55:02.210ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
97362025-08-26T21:55:02.210ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
97372025-08-26T21:55:02.210ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
97382025-08-26T21:55:02.210ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
97392025-08-26T21:55:02.210ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
97402025-08-26T21:55:02.210ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
97412025-08-26T21:55:02.210ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
97422025-08-26T21:55:02.210ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
97432025-08-26T21:55:02.210ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
97442025-08-26T21:55:02.210ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
97452025-08-26T21:55:02.211ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
97462025-08-26T21:55:02.211ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
97472025-08-26T21:55:02.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
97482025-08-26T21:55:02.213ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
97492025-08-26T21:55:02.213ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
97502025-08-26T21:55:02.331ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "ffbf3b25-0a7c-4b73-8012-1378991049a9", "date": "Tue, 26 Aug 2025 21:55:02 GMT"} })
97512025-08-26T21:55:02.331ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
97522025-08-26T21:55:02.331ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
97532025-08-26T21:55:02.338ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "f56f980b-9017-4d6e-83c9-b0056b4d9907", "content-length": "682", "date": "Tue, 26 Aug 2025 21:55:01 GMT"} })
97542025-08-26T21:55:02.338ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
97552025-08-26T21:55:02.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
97562025-08-26T21:55:02.342ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
97572025-08-26T21:55:02.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
97582025-08-26T21:55:02.346ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
97592025-08-26T21:55:02.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
97602025-08-26T21:55:02.348ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
97612025-08-26T21:55:02.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
97622025-08-26T21:55:02.355ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 681.444702ms iteration = 19 status = {}
97632025-08-26T21:55:05.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 359 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 6e9b4631-d037-471d-9d73-1f61fb00271e response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
97642025-08-26T21:55:06.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 110 reason = Timeout
97652025-08-26T21:55:06.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
97662025-08-26T21:55:06.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
97672025-08-26T21:55:06.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97682025-08-26T21:55:06.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
97692025-08-26T21:55:06.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
97702025-08-26T21:55:06.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
97712025-08-26T21:55:06.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
97722025-08-26T21:55:06.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.710112ms iteration = 110 status = {"chicken_switches_updated":false}
97732025-08-26T21:55:11.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 111 reason = Timeout
97742025-08-26T21:55:11.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
97752025-08-26T21:55:11.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
97762025-08-26T21:55:11.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97772025-08-26T21:55:11.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
97782025-08-26T21:55:11.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
97792025-08-26T21:55:11.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
97802025-08-26T21:55:11.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
97812025-08-26T21:55:11.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.398179ms iteration = 111 status = {"chicken_switches_updated":false}
97822025-08-26T21:55:11.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 56 reason = Timeout
97832025-08-26T21:55:11.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
97842025-08-26T21:55:11.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
97852025-08-26T21:55:11.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97862025-08-26T21:55:11.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
97872025-08-26T21:55:11.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
97882025-08-26T21:55:11.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
97892025-08-26T21:55:11.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
97902025-08-26T21:55:11.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
97912025-08-26T21:55:11.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97922025-08-26T21:55:11.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
97932025-08-26T21:55:11.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
97942025-08-26T21:55:11.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
97952025-08-26T21:55:11.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 56 reason = Timeout
97962025-08-26T21:55:11.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
97972025-08-26T21:55:11.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
97982025-08-26T21:55:11.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
97992025-08-26T21:55:11.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
98002025-08-26T21:55:11.689ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.481478ms iteration = 56 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
98012025-08-26T21:55:11.699ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
98022025-08-26T21:55:11.699ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.987643ms iteration = 56 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
98032025-08-26T21:55:15.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = f3ae22a2-dd38-443b-9ff8-cefa4b1c8173 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
98042025-08-26T21:55:15.345ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = f3ae22a2-dd38-443b-9ff8-cefa4b1c8173 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
98052025-08-26T21:55:15.348ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2877 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = f3ae22a2-dd38-443b-9ff8-cefa4b1c8173 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
98062025-08-26T21:55:15.348ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 3dd5ff6a-4cc0-4708-9550-ad6dadca548c roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
98072025-08-26T21:55:15.348ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 3dd5ff6a-4cc0-4708-9550-ad6dadca548c resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
98082025-08-26T21:55:15.351ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2412 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 3dd5ff6a-4cc0-4708-9550-ad6dadca548c response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
98092025-08-26T21:55:15.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 290 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 58732872-60a7-4de5-9a9c-86d13e43ae22 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
98102025-08-26T21:55:16.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 112 reason = Timeout
98112025-08-26T21:55:16.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
98122025-08-26T21:55:16.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
98132025-08-26T21:55:16.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98142025-08-26T21:55:16.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
98152025-08-26T21:55:16.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
98162025-08-26T21:55:16.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
98172025-08-26T21:55:16.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
98182025-08-26T21:55:16.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.341645ms iteration = 112 status = {"chicken_switches_updated":false}
98192025-08-26T21:55:21.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 113 reason = Timeout
98202025-08-26T21:55:21.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
98212025-08-26T21:55:21.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
98222025-08-26T21:55:21.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98232025-08-26T21:55:21.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
98242025-08-26T21:55:21.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
98252025-08-26T21:55:21.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
98262025-08-26T21:55:21.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 57 reason = Timeout
98272025-08-26T21:55:21.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
98282025-08-26T21:55:21.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
98292025-08-26T21:55:21.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
98302025-08-26T21:55:21.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.255655ms iteration = 113 status = {"chicken_switches_updated":false}
98312025-08-26T21:55:21.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98322025-08-26T21:55:21.678ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
98332025-08-26T21:55:21.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
98342025-08-26T21:55:21.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
98352025-08-26T21:55:21.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
98362025-08-26T21:55:21.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
98372025-08-26T21:55:21.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98382025-08-26T21:55:21.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
98392025-08-26T21:55:21.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
98402025-08-26T21:55:21.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
98412025-08-26T21:55:21.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 57 reason = Timeout
98422025-08-26T21:55:21.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
98432025-08-26T21:55:21.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
98442025-08-26T21:55:21.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
98452025-08-26T21:55:21.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
98462025-08-26T21:55:21.699ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
98472025-08-26T21:55:21.699ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.452664ms iteration = 57 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
98482025-08-26T21:55:21.725ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 39.078231ms iteration = 57 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
98492025-08-26T21:55:25.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 615 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 293494f5-6bb5-4fd2-84b5-4bf8ef308dc0 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
98502025-08-26T21:55:26.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 114 reason = Timeout
98512025-08-26T21:55:26.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
98522025-08-26T21:55:26.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
98532025-08-26T21:55:26.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98542025-08-26T21:55:26.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
98552025-08-26T21:55:26.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
98562025-08-26T21:55:26.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
98572025-08-26T21:55:26.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
98582025-08-26T21:55:26.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.390626ms iteration = 114 status = {"chicken_switches_updated":false}
98592025-08-26T21:55:31.664ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 20 reason = Timeout
98602025-08-26T21:55:31.664ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 20 reason = Timeout
98612025-08-26T21:55:31.664ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
98622025-08-26T21:55:31.664ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 20 reason = Timeout
98632025-08-26T21:55:31.664ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
98642025-08-26T21:55:31.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
98652025-08-26T21:55:31.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
98662025-08-26T21:55:31.665ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 115 reason = Timeout
98672025-08-26T21:55:31.665ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
98682025-08-26T21:55:31.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
98692025-08-26T21:55:31.667ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1435, tv_nsec: 525728884 } })
98702025-08-26T21:55:31.667ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1435, tv_nsec: 525997410 } })
98712025-08-26T21:55:31.667ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
98722025-08-26T21:55:31.667ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
98732025-08-26T21:55:31.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98742025-08-26T21:55:31.668ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.732034ms iteration = 20 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
98752025-08-26T21:55:31.670ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
98762025-08-26T21:55:31.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
98772025-08-26T21:55:31.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
98782025-08-26T21:55:31.672ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
98792025-08-26T21:55:31.672ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.489907ms iteration = 115 status = {"chicken_switches_updated":false}
98802025-08-26T21:55:31.674ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 21 reason = Timeout
98812025-08-26T21:55:31.674ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 20 reason = Timeout
98822025-08-26T21:55:31.674ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
98832025-08-26T21:55:31.674ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 20 reason = Timeout
98842025-08-26T21:55:31.674ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
98852025-08-26T21:55:31.674ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
98862025-08-26T21:55:31.674ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 20 reason = Timeout
98872025-08-26T21:55:31.674ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
98882025-08-26T21:55:31.674ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 20 reason = Timeout
98892025-08-26T21:55:31.674ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 20 reason = Timeout
98902025-08-26T21:55:31.674ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
98912025-08-26T21:55:31.674ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
98922025-08-26T21:55:31.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 58 reason = Timeout
98932025-08-26T21:55:31.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
98942025-08-26T21:55:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
98952025-08-26T21:55:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
98962025-08-26T21:55:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
98972025-08-26T21:55:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
98982025-08-26T21:55:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
98992025-08-26T21:55:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
99002025-08-26T21:55:31.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
99012025-08-26T21:55:31.676ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99022025-08-26T21:55:31.676ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99032025-08-26T21:55:31.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99042025-08-26T21:55:31.676ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99052025-08-26T21:55:31.679ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
99062025-08-26T21:55:31.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
99072025-08-26T21:55:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
99082025-08-26T21:55:31.680ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
99092025-08-26T21:55:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
99102025-08-26T21:55:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
99112025-08-26T21:55:31.680ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
99122025-08-26T21:55:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
99132025-08-26T21:55:31.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
99142025-08-26T21:55:31.681ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 6.689251ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
99152025-08-26T21:55:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
99162025-08-26T21:55:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
99172025-08-26T21:55:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
99182025-08-26T21:55:31.681ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
99192025-08-26T21:55:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
99202025-08-26T21:55:31.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
99212025-08-26T21:55:31.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
99222025-08-26T21:55:31.682ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1435, tv_nsec: 541685544 } })
99232025-08-26T21:55:31.682ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
99242025-08-26T21:55:31.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
99252025-08-26T21:55:31.683ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
99262025-08-26T21:55:31.683ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
99272025-08-26T21:55:31.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
99282025-08-26T21:55:31.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99292025-08-26T21:55:31.684ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99302025-08-26T21:55:31.685ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
99312025-08-26T21:55:31.685ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 20 reason = Timeout
99322025-08-26T21:55:31.685ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
99332025-08-26T21:55:31.685ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 20 reason = Timeout
99342025-08-26T21:55:31.685ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
99352025-08-26T21:55:31.685ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
99362025-08-26T21:55:31.685ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 58 reason = Timeout
99372025-08-26T21:55:31.685ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
99382025-08-26T21:55:31.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
99392025-08-26T21:55:31.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
99402025-08-26T21:55:31.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
99412025-08-26T21:55:31.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
99422025-08-26T21:55:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
99432025-08-26T21:55:31.687ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
99442025-08-26T21:55:31.687ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 20 reason = Timeout
99452025-08-26T21:55:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
99462025-08-26T21:55:31.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 20 reason = Timeout
99472025-08-26T21:55:31.687ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 15.091\u{b5}s iteration = 20 status = {"disabled":true,"errors":[],"sps":[]}
99482025-08-26T21:55:31.687ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 20 reason = Timeout
99492025-08-26T21:55:31.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
99502025-08-26T21:55:31.687ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
99512025-08-26T21:55:31.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 20 reason = Timeout
99522025-08-26T21:55:31.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
99532025-08-26T21:55:31.687ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
99542025-08-26T21:55:31.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 20 reason = Timeout
99552025-08-26T21:55:31.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
99562025-08-26T21:55:31.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 20 reason = Timeout
99572025-08-26T21:55:31.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
99582025-08-26T21:55:31.687ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 20 reason = Timeout
99592025-08-26T21:55:31.687ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
99602025-08-26T21:55:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
99612025-08-26T21:55:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
99622025-08-26T21:55:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
99632025-08-26T21:55:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
99642025-08-26T21:55:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
99652025-08-26T21:55:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
99662025-08-26T21:55:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
99672025-08-26T21:55:31.688ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
99682025-08-26T21:55:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
99692025-08-26T21:55:31.701ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99702025-08-26T21:55:31.705ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
99712025-08-26T21:55:31.706ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
99722025-08-26T21:55:31.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
99732025-08-26T21:55:31.706ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
99742025-08-26T21:55:31.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
99752025-08-26T21:55:31.712ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
99762025-08-26T21:55:31.712ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 37.933865ms iteration = 58 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
99772025-08-26T21:55:31.752ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99782025-08-26T21:55:31.756ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
99792025-08-26T21:55:31.756ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
99802025-08-26T21:55:31.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
99812025-08-26T21:55:31.759ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
99822025-08-26T21:55:31.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
99832025-08-26T21:55:31.762ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
99842025-08-26T21:55:31.762ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
99852025-08-26T21:55:31.762ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
99862025-08-26T21:55:31.762ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
99872025-08-26T21:55:31.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
99882025-08-26T21:55:31.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
99892025-08-26T21:55:31.763ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 75.751008ms iteration = 20 status = {"errors":[],"garbage_collect_requested":[]}
99902025-08-26T21:55:31.763ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
99912025-08-26T21:55:31.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
99922025-08-26T21:55:31.765ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 78.002248ms iteration = 20 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
99932025-08-26T21:55:31.767ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
99942025-08-26T21:55:31.767ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
99952025-08-26T21:55:31.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
99962025-08-26T21:55:31.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
99972025-08-26T21:55:31.768ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
99982025-08-26T21:55:31.768ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
99992025-08-26T21:55:31.769ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "445eaa17-ee5e-47b6-b77b-3cbf062404be", "content-length": "26", "date": "Tue, 26 Aug 2025 21:55:31 GMT"} })
100002025-08-26T21:55:31.769ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
100012025-08-26T21:55:31.769ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100022025-08-26T21:55:31.769ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 82.07268ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
100032025-08-26T21:55:31.771ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
100042025-08-26T21:55:31.773ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
100052025-08-26T21:55:31.773ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
100062025-08-26T21:55:31.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
100072025-08-26T21:55:31.774ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100082025-08-26T21:55:31.775ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
100092025-08-26T21:55:31.775ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
100102025-08-26T21:55:31.775ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e45f298c-3d50-49bb-a25e-4e7c825b7186", "content-length": "26", "date": "Tue, 26 Aug 2025 21:55:31 GMT"} })
100112025-08-26T21:55:31.775ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
100122025-08-26T21:55:31.775ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
100132025-08-26T21:55:31.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
100142025-08-26T21:55:31.776ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 102.437099ms iteration = 20 status = 0
100152025-08-26T21:55:31.777ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 89.853725ms iteration = 20 status = {"errors":[],"finish_invoked_ok":[]}
100162025-08-26T21:55:31.777ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
100172025-08-26T21:55:31.777ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
100182025-08-26T21:55:31.777ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
100192025-08-26T21:55:31.777ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 91.985073ms iteration = 58 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
100202025-08-26T21:55:31.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
100212025-08-26T21:55:31.780ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100222025-08-26T21:55:31.781ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100232025-08-26T21:55:31.783ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
100242025-08-26T21:55:31.783ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
100252025-08-26T21:55:31.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
100262025-08-26T21:55:31.785ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
100272025-08-26T21:55:31.785ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
100282025-08-26T21:55:31.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
100292025-08-26T21:55:31.788ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100302025-08-26T21:55:31.788ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
100312025-08-26T21:55:31.788ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 102.773417ms iteration = 20 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
100322025-08-26T21:55:31.790ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
100332025-08-26T21:55:31.790ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
100342025-08-26T21:55:31.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
100352025-08-26T21:55:31.792ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 104.812943ms iteration = 20 status = {"errors":[],"requests_created_ok":[]}
100362025-08-26T21:55:31.803ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
100372025-08-26T21:55:31.803ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
100382025-08-26T21:55:31.804ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b4f065e-8e44-4369-bb3f-0918bbde272c", "content-length": "26", "date": "Tue, 26 Aug 2025 21:55:31 GMT"} })
100392025-08-26T21:55:31.804ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
100402025-08-26T21:55:31.806ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
100412025-08-26T21:55:31.816ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
100422025-08-26T21:55:31.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
100432025-08-26T21:55:31.819ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100442025-08-26T21:55:31.822ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
100452025-08-26T21:55:31.822ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
100462025-08-26T21:55:31.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
100472025-08-26T21:55:31.826ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
100482025-08-26T21:55:31.827ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
100492025-08-26T21:55:31.827ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
100502025-08-26T21:55:31.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
100512025-08-26T21:55:31.828ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
100522025-08-26T21:55:31.828ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 153.52664ms iteration = 20 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
100532025-08-26T21:55:31.870ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 196.485139ms iteration = 21 status = {"physical_disks_added":0}
100542025-08-26T21:55:31.882ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
100552025-08-26T21:55:31.882ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
100562025-08-26T21:55:31.882ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 196.999441ms iteration = 20 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
100572025-08-26T21:55:31.931ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 267.435099ms iteration = 20 status = {}
100582025-08-26T21:55:31.953ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
100592025-08-26T21:55:31.957ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "8078625d-a990-4bd5-a23f-9f60c7cc5e0c", "content-length": "691", "date": "Tue, 26 Aug 2025 21:55:31 GMT"} })
100602025-08-26T21:55:31.957ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
100612025-08-26T21:55:31.957ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
100622025-08-26T21:55:31.957ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
100632025-08-26T21:55:31.959ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 284.633246ms iteration = 20 status = {}
100642025-08-26T21:55:31.988ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
100652025-08-26T21:55:31.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
100662025-08-26T21:55:31.991ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 327.154355ms iteration = 20 status = null
100672025-08-26T21:55:32.006ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
100682025-08-26T21:55:32.006ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
100692025-08-26T21:55:32.006ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "789bdfd5-f189-423c-80d4-a26814e59b6b", "content-length": "372", "date": "Tue, 26 Aug 2025 21:55:31 GMT"} })
100702025-08-26T21:55:32.006ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
100712025-08-26T21:55:32.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
100722025-08-26T21:55:32.056ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
100732025-08-26T21:55:32.056ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
100742025-08-26T21:55:32.056ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
100752025-08-26T21:55:32.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
100762025-08-26T21:55:32.056ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "d925e88b-b1ec-4eee-bb4a-ab8acc403a6f", "date": "Tue, 26 Aug 2025 21:55:31 GMT"} })
100772025-08-26T21:55:32.056ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
100782025-08-26T21:55:32.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
100792025-08-26T21:55:32.058ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
100802025-08-26T21:55:32.058ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
100812025-08-26T21:55:32.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
100822025-08-26T21:55:32.059ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
100832025-08-26T21:55:32.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
100842025-08-26T21:55:32.062ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
100852025-08-26T21:55:32.062ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
100862025-08-26T21:55:32.065ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
100872025-08-26T21:55:32.065ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
100882025-08-26T21:55:32.066ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "bc5f90fd-0015-4356-b226-dd0eee7bce23", "date": "Tue, 26 Aug 2025 21:55:32 GMT"} })
100892025-08-26T21:55:32.067ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 379.943455ms iteration = 20 status = {}
100902025-08-26T21:55:32.070ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
100912025-08-26T21:55:32.070ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
100922025-08-26T21:55:32.071ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
100932025-08-26T21:55:32.071ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
100942025-08-26T21:55:32.071ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
100952025-08-26T21:55:32.071ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
100962025-08-26T21:55:32.071ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
100972025-08-26T21:55:32.071ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
100982025-08-26T21:55:32.071ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
100992025-08-26T21:55:32.071ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
101002025-08-26T21:55:32.071ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
101012025-08-26T21:55:32.071ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
101022025-08-26T21:55:32.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
101032025-08-26T21:55:32.074ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
101042025-08-26T21:55:32.074ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
101052025-08-26T21:55:32.234ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "5fa3b468-ef55-48fd-9de8-4437c6f7b561", "date": "Tue, 26 Aug 2025 21:55:31 GMT"} })
101062025-08-26T21:55:32.234ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
101072025-08-26T21:55:32.234ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
101082025-08-26T21:55:32.235ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "91cadfbc-ad86-4d52-b4f6-f72ed83ca6ae", "content-length": "682", "date": "Tue, 26 Aug 2025 21:55:32 GMT"} })
101092025-08-26T21:55:32.235ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
101102025-08-26T21:55:32.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
101112025-08-26T21:55:32.238ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
101122025-08-26T21:55:32.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
101132025-08-26T21:55:32.242ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
101142025-08-26T21:55:32.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
101152025-08-26T21:55:32.244ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
101162025-08-26T21:55:32.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
101172025-08-26T21:55:32.251ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 577.392206ms iteration = 20 status = {}
101182025-08-26T21:55:35.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 393 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = e22b79fc-0bbf-488d-9f00-66954dc995bf response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
101192025-08-26T21:55:36.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 116 reason = Timeout
101202025-08-26T21:55:36.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
101212025-08-26T21:55:36.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
101222025-08-26T21:55:36.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101232025-08-26T21:55:36.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
101242025-08-26T21:55:36.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
101252025-08-26T21:55:36.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
101262025-08-26T21:55:36.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
101272025-08-26T21:55:36.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.689243ms iteration = 116 status = {"chicken_switches_updated":false}
101282025-08-26T21:55:41.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 117 reason = Timeout
101292025-08-26T21:55:41.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
101302025-08-26T21:55:41.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
101312025-08-26T21:55:41.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101322025-08-26T21:55:41.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
101332025-08-26T21:55:41.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
101342025-08-26T21:55:41.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
101352025-08-26T21:55:41.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
101362025-08-26T21:55:41.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.721082ms iteration = 117 status = {"chicken_switches_updated":false}
101372025-08-26T21:55:41.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 59 reason = Timeout
101382025-08-26T21:55:41.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
101392025-08-26T21:55:41.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
101402025-08-26T21:55:41.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101412025-08-26T21:55:41.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
101422025-08-26T21:55:41.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
101432025-08-26T21:55:41.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
101442025-08-26T21:55:41.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
101452025-08-26T21:55:41.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
101462025-08-26T21:55:41.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101472025-08-26T21:55:41.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 59 reason = Timeout
101482025-08-26T21:55:41.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
101492025-08-26T21:55:41.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
101502025-08-26T21:55:41.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
101512025-08-26T21:55:41.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
101522025-08-26T21:55:41.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
101532025-08-26T21:55:41.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
101542025-08-26T21:55:41.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
101552025-08-26T21:55:41.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.589801ms iteration = 59 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
101562025-08-26T21:55:41.699ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
101572025-08-26T21:55:41.699ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.926062ms iteration = 59 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
101582025-08-26T21:55:45.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 17f79b3b-71e4-422c-b96b-710bf861f56b roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
101592025-08-26T21:55:45.345ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 17f79b3b-71e4-422c-b96b-710bf861f56b resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
101602025-08-26T21:55:45.348ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3379 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 17f79b3b-71e4-422c-b96b-710bf861f56b response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
101612025-08-26T21:55:45.349ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 60375a88-902b-48a2-9210-0b9ffd4930df roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
101622025-08-26T21:55:45.349ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 60375a88-902b-48a2-9210-0b9ffd4930df resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
101632025-08-26T21:55:45.352ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3097 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59936 req_id = 60375a88-902b-48a2-9210-0b9ffd4930df response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
101642025-08-26T21:55:45.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 286 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = a614cbc0-ba36-4fa6-bc65-15335c3f6eaf response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
101652025-08-26T21:55:46.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 118 reason = Timeout
101662025-08-26T21:55:46.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
101672025-08-26T21:55:46.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
101682025-08-26T21:55:46.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101692025-08-26T21:55:46.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
101702025-08-26T21:55:46.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
101712025-08-26T21:55:46.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
101722025-08-26T21:55:46.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
101732025-08-26T21:55:46.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.554347ms iteration = 118 status = {"chicken_switches_updated":false}
101742025-08-26T21:55:51.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 119 reason = Timeout
101752025-08-26T21:55:51.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
101762025-08-26T21:55:51.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
101772025-08-26T21:55:51.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101782025-08-26T21:55:51.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
101792025-08-26T21:55:51.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
101802025-08-26T21:55:51.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
101812025-08-26T21:55:51.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
101822025-08-26T21:55:51.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.003328ms iteration = 119 status = {"chicken_switches_updated":false}
101832025-08-26T21:55:51.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 60 reason = Timeout
101842025-08-26T21:55:51.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
101852025-08-26T21:55:51.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
101862025-08-26T21:55:51.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101872025-08-26T21:55:51.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
101882025-08-26T21:55:51.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
101892025-08-26T21:55:51.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
101902025-08-26T21:55:51.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
101912025-08-26T21:55:51.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
101922025-08-26T21:55:51.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101932025-08-26T21:55:51.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
101942025-08-26T21:55:51.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
101952025-08-26T21:55:51.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
101962025-08-26T21:55:51.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 60 reason = Timeout
101972025-08-26T21:55:51.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
101982025-08-26T21:55:51.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
101992025-08-26T21:55:51.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
102002025-08-26T21:55:51.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
102012025-08-26T21:55:51.689ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.454008ms iteration = 60 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
102022025-08-26T21:55:51.698ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
102032025-08-26T21:55:51.698ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.245486ms iteration = 60 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
102042025-08-26T21:55:55.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 393 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 3aa3cb5e-c62a-47f4-b29e-3c94fd33cb9b response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
102052025-08-26T21:55:56.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 120 reason = Timeout
102062025-08-26T21:55:56.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
102072025-08-26T21:55:56.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
102082025-08-26T21:55:56.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102092025-08-26T21:55:56.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
102102025-08-26T21:55:56.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
102112025-08-26T21:55:56.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
102122025-08-26T21:55:56.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
102132025-08-26T21:55:56.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.311642ms iteration = 120 status = {"chicken_switches_updated":false}
102142025-08-26T21:56:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
102152025-08-26T21:56:01.664ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
102162025-08-26T21:56:01.664ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
102172025-08-26T21:56:01.664ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 11 reason = Timeout
102182025-08-26T21:56:01.664ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:46:01.664612553 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
102192025-08-26T21:56:01.664ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
102202025-08-26T21:56:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
102212025-08-26T21:56:01.664ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 21 reason = Timeout
102222025-08-26T21:56:01.664ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 21 reason = Timeout
102232025-08-26T21:56:01.664ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
102242025-08-26T21:56:01.664ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 21 reason = Timeout
102252025-08-26T21:56:01.664ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
102262025-08-26T21:56:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
102272025-08-26T21:56:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
102282025-08-26T21:56:01.664ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
102292025-08-26T21:56:01.664ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
102302025-08-26T21:56:01.664ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
102312025-08-26T21:56:01.664ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
102322025-08-26T21:56:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
102332025-08-26T21:56:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
102342025-08-26T21:56:01.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
102352025-08-26T21:56:01.665ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 12 reason = Timeout
102362025-08-26T21:56:01.665ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
102372025-08-26T21:56:01.665ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 15.37\u{b5}s iteration = 12 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
102382025-08-26T21:56:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
102392025-08-26T21:56:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
102402025-08-26T21:56:01.665ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1465, tv_nsec: 523835331 } })
102412025-08-26T21:56:01.665ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1465, tv_nsec: 523830122 } })
102422025-08-26T21:56:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
102432025-08-26T21:56:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
102442025-08-26T21:56:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
102452025-08-26T21:56:01.665ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
102462025-08-26T21:56:01.665ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 905.081\u{b5}s iteration = 11 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
102472025-08-26T21:56:01.665ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
102482025-08-26T21:56:01.665ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.077004ms iteration = 11 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
102492025-08-26T21:56:01.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
102502025-08-26T21:56:01.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
102512025-08-26T21:56:01.665ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 2 reason = Timeout
102522025-08-26T21:56:01.665ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
102532025-08-26T21:56:01.665ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 121 reason = Timeout
102542025-08-26T21:56:01.665ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
102552025-08-26T21:56:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
102562025-08-26T21:56:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
102572025-08-26T21:56:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
102582025-08-26T21:56:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
102592025-08-26T21:56:01.666ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102602025-08-26T21:56:01.667ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1465, tv_nsec: 525885998 } })
102612025-08-26T21:56:01.667ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:46:01.664612553 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
102622025-08-26T21:56:01.667ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 2.854734ms iteration = 11 status = {"expiration":"2025-08-26T21:46:01.664612553Z","pruned":[]}
102632025-08-26T21:56:01.667ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
102642025-08-26T21:56:01.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102652025-08-26T21:56:01.667ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102662025-08-26T21:56:01.668ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1465, tv_nsec: 526706336 } })
102672025-08-26T21:56:01.668ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102682025-08-26T21:56:01.668ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102692025-08-26T21:56:01.668ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
102702025-08-26T21:56:01.668ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.313888ms iteration = 21 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
102712025-08-26T21:56:01.670ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
102722025-08-26T21:56:01.670ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
102732025-08-26T21:56:01.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
102742025-08-26T21:56:01.671ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
102752025-08-26T21:56:01.671ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
102762025-08-26T21:56:01.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
102772025-08-26T21:56:01.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
102782025-08-26T21:56:01.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
102792025-08-26T21:56:01.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
102802025-08-26T21:56:01.673ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
102812025-08-26T21:56:01.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
102822025-08-26T21:56:01.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
102832025-08-26T21:56:01.673ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 13 reason = Timeout
102842025-08-26T21:56:01.673ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 21 reason = Timeout
102852025-08-26T21:56:01.673ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
102862025-08-26T21:56:01.673ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 11.421\u{b5}s iteration = 13 status = "Disabled"
102872025-08-26T21:56:01.673ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
102882025-08-26T21:56:01.674ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 22 reason = Timeout
102892025-08-26T21:56:01.674ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
102902025-08-26T21:56:01.674ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 21 reason = Timeout
102912025-08-26T21:56:01.674ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
102922025-08-26T21:56:01.674ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 11 reason = Timeout
102932025-08-26T21:56:01.674ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
102942025-08-26T21:56:01.674ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
102952025-08-26T21:56:01.674ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 21 reason = Timeout
102962025-08-26T21:56:01.674ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
102972025-08-26T21:56:01.674ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 21 reason = Timeout
102982025-08-26T21:56:01.674ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
102992025-08-26T21:56:01.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 61 reason = Timeout
103002025-08-26T21:56:01.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
103012025-08-26T21:56:01.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 4 reason = Timeout
103022025-08-26T21:56:01.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
103032025-08-26T21:56:01.674ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 21 reason = Timeout
103042025-08-26T21:56:01.674ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
103052025-08-26T21:56:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
103062025-08-26T21:56:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
103072025-08-26T21:56:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
103082025-08-26T21:56:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
103092025-08-26T21:56:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
103102025-08-26T21:56:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
103112025-08-26T21:56:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
103122025-08-26T21:56:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
103132025-08-26T21:56:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
103142025-08-26T21:56:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
103152025-08-26T21:56:01.675ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
103162025-08-26T21:56:01.675ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 9.063374ms iteration = 121 status = {"chicken_switches_updated":false}
103172025-08-26T21:56:01.675ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103182025-08-26T21:56:01.675ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103192025-08-26T21:56:01.675ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103202025-08-26T21:56:01.676ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
103212025-08-26T21:56:01.676ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
103222025-08-26T21:56:01.676ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
103232025-08-26T21:56:01.676ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
103242025-08-26T21:56:01.676ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103252025-08-26T21:56:01.676ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103262025-08-26T21:56:01.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103272025-08-26T21:56:01.676ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103282025-08-26T21:56:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
103292025-08-26T21:56:01.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103302025-08-26T21:56:01.678ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103312025-08-26T21:56:01.679ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(0dd3b372-00f5-4ddd-acec-8c8a4f49c797, 0), (98fcdfec-5acc-4b21-be95-18fc820aa3d1, 0), (f37c741e-e5eb-4be5-8e12-bbe92da18b97, 0), (22ce40dd-e0b9-460a-8151-60c0f8e6fa4f, 0)] collection_id = 0dd3b372-00f5-4ddd-acec-8c8a4f49c797
103322025-08-26T21:56:01.679ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
103332025-08-26T21:56:01.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
103342025-08-26T21:56:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
103352025-08-26T21:56:01.682ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
103362025-08-26T21:56:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
103372025-08-26T21:56:01.682ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
103382025-08-26T21:56:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
103392025-08-26T21:56:01.682ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
103402025-08-26T21:56:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
103412025-08-26T21:56:01.682ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
103422025-08-26T21:56:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
103432025-08-26T21:56:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
103442025-08-26T21:56:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
103452025-08-26T21:56:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
103462025-08-26T21:56:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
103472025-08-26T21:56:01.683ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
103482025-08-26T21:56:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
103492025-08-26T21:56:01.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
103502025-08-26T21:56:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
103512025-08-26T21:56:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
103522025-08-26T21:56:01.684ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
103532025-08-26T21:56:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
103542025-08-26T21:56:01.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
103552025-08-26T21:56:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
103562025-08-26T21:56:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
103572025-08-26T21:56:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
103582025-08-26T21:56:01.686ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
103592025-08-26T21:56:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
103602025-08-26T21:56:01.686ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
103612025-08-26T21:56:01.686ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
103622025-08-26T21:56:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
103632025-08-26T21:56:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
103642025-08-26T21:56:01.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
103652025-08-26T21:56:01.686ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 3 reason = Timeout
103662025-08-26T21:56:01.686ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
103672025-08-26T21:56:01.686ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 21 reason = Timeout
103682025-08-26T21:56:01.686ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
103692025-08-26T21:56:01.686ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
103702025-08-26T21:56:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
103712025-08-26T21:56:01.686ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 11 reason = Timeout
103722025-08-26T21:56:01.686ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
103732025-08-26T21:56:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
103742025-08-26T21:56:01.686ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 21 reason = Timeout
103752025-08-26T21:56:01.686ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
103762025-08-26T21:56:01.686ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
103772025-08-26T21:56:01.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 61 reason = Timeout
103782025-08-26T21:56:01.686ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
103792025-08-26T21:56:01.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
103802025-08-26T21:56:01.686ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 21 reason = Timeout
103812025-08-26T21:56:01.686ZDEBGnexus (BackgroundTasks): activating background_task = saga_recovery iteration = 2 reason = Timeout
103822025-08-26T21:56:01.687ZINFOnexus (SagaQuiesceHandle): saga recovery pass starting file = nexus/types/src/quiesce.rs:319
103832025-08-26T21:56:01.687ZDEBGnexus (SagaRecoverer): listing candidate sagas for recovery
103842025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 12.41\u{b5}s iteration = 21 status = {"disabled":true,"errors":[],"sps":[]}
103852025-08-26T21:56:01.687ZDEBGnexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
103862025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 21 reason = Timeout
103872025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 11 reason = Timeout
103882025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
103892025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
103902025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 21 reason = Timeout
103912025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
103922025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
103932025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 11 reason = Timeout
103942025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
103952025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103962025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 11 reason = Timeout
103972025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
103982025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 3 reason = Timeout
103992025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 21 reason = Timeout
104002025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
104012025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
104022025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 21 reason = Timeout
104032025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
104042025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 21 reason = Timeout
104052025-08-26T21:56:01.687ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
104062025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
104072025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 11 reason = Timeout
104082025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 21 reason = Timeout
104092025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
104102025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
104112025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
104122025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
104132025-08-26T21:56:01.687ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1465, tv_nsec: 545985901 } })
104142025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
104152025-08-26T21:56:01.687ZDEBGnexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
104162025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
104172025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
104182025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
104192025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
104202025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
104212025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
104222025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
104232025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
104242025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
104252025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
104262025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
104272025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
104282025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
104292025-08-26T21:56:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
104302025-08-26T21:56:01.688ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
104312025-08-26T21:56:01.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
104322025-08-26T21:56:01.689ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104332025-08-26T21:56:01.689ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 2.756033ms iteration = 21 status = {"errors":[],"finish_invoked_ok":[]}
104342025-08-26T21:56:01.690ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104352025-08-26T21:56:01.690ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
104362025-08-26T21:56:01.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
104372025-08-26T21:56:01.691ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
104382025-08-26T21:56:01.691ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
104392025-08-26T21:56:01.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
104402025-08-26T21:56:01.691ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104412025-08-26T21:56:01.693ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
104422025-08-26T21:56:01.693ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
104432025-08-26T21:56:01.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
104442025-08-26T21:56:01.695ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
104452025-08-26T21:56:01.695ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
104462025-08-26T21:56:01.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
104472025-08-26T21:56:01.695ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
104482025-08-26T21:56:01.695ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
104492025-08-26T21:56:01.695ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
104502025-08-26T21:56:01.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
104512025-08-26T21:56:01.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
104522025-08-26T21:56:01.726ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
104532025-08-26T21:56:01.726ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 39.51334ms iteration = 11 status = {"by_rx":{},"error":null}
104542025-08-26T21:56:01.759ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
104552025-08-26T21:56:01.759ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
104562025-08-26T21:56:01.759ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
104572025-08-26T21:56:01.759ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
104582025-08-26T21:56:01.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
104592025-08-26T21:56:01.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104602025-08-26T21:56:01.766ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104612025-08-26T21:56:01.766ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 79.249636ms iteration = 11 status = {"errors":[],"found_port_ok":[]}
104622025-08-26T21:56:01.768ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
104632025-08-26T21:56:01.768ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
104642025-08-26T21:56:01.768ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
104652025-08-26T21:56:01.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
104662025-08-26T21:56:01.769ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
104672025-08-26T21:56:01.769ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
104682025-08-26T21:56:01.769ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 95.711668ms iteration = 11 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
104692025-08-26T21:56:01.769ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
104702025-08-26T21:56:01.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
104712025-08-26T21:56:01.770ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
104722025-08-26T21:56:01.770ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
104732025-08-26T21:56:01.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
104742025-08-26T21:56:01.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
104752025-08-26T21:56:01.770ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
104762025-08-26T21:56:01.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
104772025-08-26T21:56:01.771ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
104782025-08-26T21:56:01.772ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
104792025-08-26T21:56:01.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
104802025-08-26T21:56:01.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
104812025-08-26T21:56:01.774ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
104822025-08-26T21:56:01.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
104832025-08-26T21:56:01.777ZINFOnexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:473
104842025-08-26T21:56:01.777ZINFOnexus (SagaQuiesceHandle): saga recovery pass finished file = nexus/types/src/quiesce.rs:332 generation = 1
104852025-08-26T21:56:01.777ZDEBGnexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 90.177752ms iteration = 2 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
104862025-08-26T21:56:01.778ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
104872025-08-26T21:56:01.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
104882025-08-26T21:56:01.782ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
104892025-08-26T21:56:01.782ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
104902025-08-26T21:56:01.782ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 95.621236ms iteration = 11 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
104912025-08-26T21:56:01.788ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 101.968759ms iteration = 21 status = {"errors":[],"garbage_collect_requested":[]}
104922025-08-26T21:56:01.789ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104932025-08-26T21:56:01.789ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104942025-08-26T21:56:01.795ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
104952025-08-26T21:56:01.795ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
104962025-08-26T21:56:01.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
104972025-08-26T21:56:01.795ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
104982025-08-26T21:56:01.795ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
104992025-08-26T21:56:01.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
105002025-08-26T21:56:01.797ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105012025-08-26T21:56:01.800ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
105022025-08-26T21:56:01.800ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
105032025-08-26T21:56:01.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
105042025-08-26T21:56:01.803ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
105052025-08-26T21:56:01.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
105062025-08-26T21:56:01.803ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 129.763375ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
105072025-08-26T21:56:01.804ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
105082025-08-26T21:56:01.804ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
105092025-08-26T21:56:01.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
105102025-08-26T21:56:01.804ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 117.708863ms iteration = 21 status = {"errors":[],"requests_created_ok":[]}
105112025-08-26T21:56:01.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
105122025-08-26T21:56:01.807ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 120.664169ms iteration = 11 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
105132025-08-26T21:56:01.811ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
105142025-08-26T21:56:01.811ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 124.337453ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
105152025-08-26T21:56:01.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
105162025-08-26T21:56:01.812ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
105172025-08-26T21:56:01.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
105182025-08-26T21:56:01.814ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 127.196517ms iteration = 61 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
105192025-08-26T21:56:01.814ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105202025-08-26T21:56:01.814ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105212025-08-26T21:56:01.816ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
105222025-08-26T21:56:01.816ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
105232025-08-26T21:56:01.816ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-08-26 21:46:04.065217 UTC dns_group = internal generation = 1
105242025-08-26T21:56:01.816ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 151.519315ms iteration = 11 status = {"generation":1}
105252025-08-26T21:56:01.819ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
105262025-08-26T21:56:01.819ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
105272025-08-26T21:56:01.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
105282025-08-26T21:56:01.820ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
105292025-08-26T21:56:01.821ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
105302025-08-26T21:56:01.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
105312025-08-26T21:56:01.823ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 149.553951ms iteration = 21 status = 0
105322025-08-26T21:56:01.834ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
105332025-08-26T21:56:01.834ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 160.262522ms iteration = 61 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
105342025-08-26T21:56:01.845ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
105352025-08-26T21:56:01.845ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
105362025-08-26T21:56:01.846ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4147375-6ac5-4bf0-9287-8b9f83afc034", "content-length": "26", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
105372025-08-26T21:56:01.846ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
105382025-08-26T21:56:01.848ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
105392025-08-26T21:56:01.849ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
105402025-08-26T21:56:01.858ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
105412025-08-26T21:56:01.858ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 184.10037ms iteration = 21 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
105422025-08-26T21:56:01.863ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
105432025-08-26T21:56:01.863ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
105442025-08-26T21:56:01.863ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
105452025-08-26T21:56:01.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
105462025-08-26T21:56:01.873ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
105472025-08-26T21:56:01.873ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
105482025-08-26T21:56:01.873ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
105492025-08-26T21:56:01.873ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
105502025-08-26T21:56:01.873ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 187.059677ms iteration = 21 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
105512025-08-26T21:56:01.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
105522025-08-26T21:56:01.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
105532025-08-26T21:56:01.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
105542025-08-26T21:56:01.880ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
105552025-08-26T21:56:01.880ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
105562025-08-26T21:56:01.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105572025-08-26T21:56:01.880ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 193.542863ms iteration = 21 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
105582025-08-26T21:56:01.883ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 195.963228ms iteration = 21 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
105592025-08-26T21:56:01.883ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
105602025-08-26T21:56:01.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
105612025-08-26T21:56:01.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
105622025-08-26T21:56:01.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105632025-08-26T21:56:01.888ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
105642025-08-26T21:56:01.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
105652025-08-26T21:56:01.889ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
105662025-08-26T21:56:01.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
105672025-08-26T21:56:01.889ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
105682025-08-26T21:56:01.889ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 0 version = 2
105692025-08-26T21:56:01.889ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
105702025-08-26T21:56:01.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
105712025-08-26T21:56:01.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
105722025-08-26T21:56:01.890ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
105732025-08-26T21:56:01.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
105742025-08-26T21:56:01.891ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105752025-08-26T21:56:01.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
105762025-08-26T21:56:01.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
105772025-08-26T21:56:01.894ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105782025-08-26T21:56:01.895ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105792025-08-26T21:56:01.897ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
105802025-08-26T21:56:01.897ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
105812025-08-26T21:56:01.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
105822025-08-26T21:56:01.897ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
105832025-08-26T21:56:01.897ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
105842025-08-26T21:56:01.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
105852025-08-26T21:56:01.898ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
105862025-08-26T21:56:01.898ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
105872025-08-26T21:56:01.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
105882025-08-26T21:56:01.900ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 4 version = 2
105892025-08-26T21:56:01.900ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
105902025-08-26T21:56:01.900ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-08-26 21:46:05.296205 UTC dns_group = external generation = 2
105912025-08-26T21:56:01.900ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 235.707673ms iteration = 11 status = {"generation":2}
105922025-08-26T21:56:01.908ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105932025-08-26T21:56:01.910ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
105942025-08-26T21:56:01.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
105952025-08-26T21:56:01.912ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
105962025-08-26T21:56:01.912ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
105972025-08-26T21:56:01.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
105982025-08-26T21:56:01.913ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
105992025-08-26T21:56:01.913ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
106002025-08-26T21:56:01.914ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e3a05e6-f63c-42a7-ae32-c5293878bd92", "content-length": "26", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
106012025-08-26T21:56:01.914ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
106022025-08-26T21:56:01.915ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
106032025-08-26T21:56:01.917ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:113 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
106042025-08-26T21:56:01.917ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 243.09387ms iteration = 4 status = {"blueprint_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","inventory_collection_id":"22ce40dd-e0b9-460a-8151-60c0f8e6fa4f","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
106052025-08-26T21:56:01.923ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
106062025-08-26T21:56:01.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
106072025-08-26T21:56:01.926ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106082025-08-26T21:56:01.929ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
106092025-08-26T21:56:01.929ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
106102025-08-26T21:56:01.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
106112025-08-26T21:56:01.931ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106122025-08-26T21:56:01.934ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
106132025-08-26T21:56:01.934ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
106142025-08-26T21:56:01.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
106152025-08-26T21:56:01.937ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 0dd3b372-00f5-4ddd-acec-8c8a4f49c797 file = nexus/db-queries/src/db/datastore/inventory.rs:2266 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
106162025-08-26T21:56:01.937ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
106172025-08-26T21:56:01.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
106182025-08-26T21:56:01.941ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(98fcdfec-5acc-4b21-be95-18fc820aa3d1, 0), (f37c741e-e5eb-4be5-8e12-bbe92da18b97, 0), (22ce40dd-e0b9-460a-8151-60c0f8e6fa4f, 0)]
106192025-08-26T21:56:01.942ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1465, tv_nsec: 801016190 } })
106202025-08-26T21:56:01.956ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
106212025-08-26T21:56:01.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
106222025-08-26T21:56:01.958ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
106232025-08-26T21:56:01.958ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
106242025-08-26T21:56:01.959ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb3c0ee5-7ea0-4b9f-a76e-38b3da981060", "content-length": "26", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
106252025-08-26T21:56:01.959ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
106262025-08-26T21:56:01.961ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
106272025-08-26T21:56:01.964ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106282025-08-26T21:56:01.969ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
106292025-08-26T21:56:01.969ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106302025-08-26T21:56:01.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106312025-08-26T21:56:01.971ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106322025-08-26T21:56:01.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106332025-08-26T21:56:01.973ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106342025-08-26T21:56:01.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106352025-08-26T21:56:01.975ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106362025-08-26T21:56:01.977ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-08-26T21:45:59.866217Z, time_modified: 2025-08-26T21:45:59.866217Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }) }
106372025-08-26T21:56:01.977ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 312.454674ms iteration = 11 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]}},"default_endpoint":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
106382025-08-26T21:56:01.979ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
106392025-08-26T21:56:01.979ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106402025-08-26T21:56:01.979ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106412025-08-26T21:56:01.987ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106422025-08-26T21:56:01.987ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106432025-08-26T21:56:01.990ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106442025-08-26T21:56:01.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106452025-08-26T21:56:01.995ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106462025-08-26T21:56:01.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106472025-08-26T21:56:02.000ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106482025-08-26T21:56:02.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106492025-08-26T21:56:02.003ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
106502025-08-26T21:56:02.003ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106512025-08-26T21:56:02.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106522025-08-26T21:56:02.006ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106532025-08-26T21:56:02.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106542025-08-26T21:56:02.008ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106552025-08-26T21:56:02.011ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 337.282053ms iteration = 22 status = {"physical_disks_added":0}
106562025-08-26T21:56:02.012ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
106572025-08-26T21:56:02.012ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106582025-08-26T21:56:02.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106592025-08-26T21:56:02.015ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106602025-08-26T21:56:02.015ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106612025-08-26T21:56:02.017ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106622025-08-26T21:56:02.017ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106632025-08-26T21:56:02.020ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106642025-08-26T21:56:02.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106652025-08-26T21:56:02.024ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106662025-08-26T21:56:02.024ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106672025-08-26T21:56:02.025ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106682025-08-26T21:56:02.026ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106692025-08-26T21:56:02.027ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106702025-08-26T21:56:02.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106712025-08-26T21:56:02.029ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106722025-08-26T21:56:02.037ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
106732025-08-26T21:56:02.037ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106742025-08-26T21:56:02.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106752025-08-26T21:56:02.039ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106762025-08-26T21:56:02.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106772025-08-26T21:56:02.041ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106782025-08-26T21:56:02.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106792025-08-26T21:56:02.043ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106802025-08-26T21:56:02.047ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
106812025-08-26T21:56:02.047ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106822025-08-26T21:56:02.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106832025-08-26T21:56:02.054ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106842025-08-26T21:56:02.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106852025-08-26T21:56:02.055ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106862025-08-26T21:56:02.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106872025-08-26T21:56:02.058ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106882025-08-26T21:56:02.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106892025-08-26T21:56:02.060ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106902025-08-26T21:56:02.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106912025-08-26T21:56:02.063ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106922025-08-26T21:56:02.064ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106932025-08-26T21:56:02.066ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106942025-08-26T21:56:02.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106952025-08-26T21:56:02.068ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
106962025-08-26T21:56:02.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
106972025-08-26T21:56:02.070ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106982025-08-26T21:56:02.076ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
106992025-08-26T21:56:02.076ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
107002025-08-26T21:56:02.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
107012025-08-26T21:56:02.078ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
107022025-08-26T21:56:02.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
107032025-08-26T21:56:02.079ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
107042025-08-26T21:56:02.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
107052025-08-26T21:56:02.081ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107062025-08-26T21:56:02.086ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
107072025-08-26T21:56:02.086ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
107082025-08-26T21:56:02.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
107092025-08-26T21:56:02.095ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 0106180c-e6f8-4d65-969e-4fbbddf596a1, kind: Service { id: 60abf731-31e1-4111-a3e2-65da89eaed87 }, name: Name("external-dns-60abf731-31e1-4111-a3e2-65da89eaed87"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 81])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b69ae953-8f67-4cb3-9521-db9e707a6f2d, kind: Service { id: 84c849eb-0949-41d9-ad02-9a5509144564 }, name: Name("external-dns-84c849eb-0949-41d9-ad02-9a5509144564"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 136])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 0802405a-a5f2-402c-94da-ccafb58bccb5, kind: Service { id: 4996dcc4-c530-4f64-b407-fa372462bd9e }, name: Name("nexus-4996dcc4-c530-4f64-b407-fa372462bd9e"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 1acb6bfd-558b-4271-bbf5-58e0d089b710, kind: Service { id: 0205eaf0-3fc6-4ad5-b09b-732af233e9c1 }, name: Name("nexus-0205eaf0-3fc6-4ad5-b09b-732af233e9c1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 142])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cbb99fa2-edc6-497d-9c47-8dd2642f8945, kind: Service { id: 5d0959e5-47a6-4bf3-a52b-57950b9d900d }, name: Name("nexus-5d0959e5-47a6-4bf3-a52b-57950b9d900d"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 80])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
107102025-08-26T21:56:02.096ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
107112025-08-26T21:56:02.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
107122025-08-26T21:56:02.098ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107132025-08-26T21:56:02.101ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
107142025-08-26T21:56:02.101ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
107152025-08-26T21:56:02.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
107162025-08-26T21:56:02.104ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
107172025-08-26T21:56:02.104ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0106180c-e6f8-4d65-969e-4fbbddf596a1, kind: Service { id: 60abf731-31e1-4111-a3e2-65da89eaed87 }, name: Name("external-dns-60abf731-31e1-4111-a3e2-65da89eaed87"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 81])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b69ae953-8f67-4cb3-9521-db9e707a6f2d, kind: Service { id: 84c849eb-0949-41d9-ad02-9a5509144564 }, name: Name("external-dns-84c849eb-0949-41d9-ad02-9a5509144564"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 136])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0802405a-a5f2-402c-94da-ccafb58bccb5, kind: Service { id: 4996dcc4-c530-4f64-b407-fa372462bd9e }, name: Name("nexus-4996dcc4-c530-4f64-b407-fa372462bd9e"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 1acb6bfd-558b-4271-bbf5-58e0d089b710, kind: Service { id: 0205eaf0-3fc6-4ad5-b09b-732af233e9c1 }, name: Name("nexus-0205eaf0-3fc6-4ad5-b09b-732af233e9c1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 142])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cbb99fa2-edc6-497d-9c47-8dd2642f8945, kind: Service { id: 5d0959e5-47a6-4bf3-a52b-57950b9d900d }, name: Name("nexus-5d0959e5-47a6-4bf3-a52b-57950b9d900d"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 80])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 0106180c-e6f8-4d65-969e-4fbbddf596a1, kind: Service { id: 60abf731-31e1-4111-a3e2-65da89eaed87 }, name: Name("external-dns-60abf731-31e1-4111-a3e2-65da89eaed87"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 156, 81])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b69ae953-8f67-4cb3-9521-db9e707a6f2d, kind: Service { id: 84c849eb-0949-41d9-ad02-9a5509144564 }, name: Name("external-dns-84c849eb-0949-41d9-ad02-9a5509144564"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 136])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0802405a-a5f2-402c-94da-ccafb58bccb5, kind: Service { id: 4996dcc4-c530-4f64-b407-fa372462bd9e }, name: Name("nexus-4996dcc4-c530-4f64-b407-fa372462bd9e"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 1acb6bfd-558b-4271-bbf5-58e0d089b710, kind: Service { id: 0205eaf0-3fc6-4ad5-b09b-732af233e9c1 }, name: Name("nexus-0205eaf0-3fc6-4ad5-b09b-732af233e9c1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 142])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cbb99fa2-edc6-497d-9c47-8dd2642f8945, kind: Service { id: 5d0959e5-47a6-4bf3-a52b-57950b9d900d }, name: Name("nexus-5d0959e5-47a6-4bf3-a52b-57950b9d900d"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 80])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
107182025-08-26T21:56:02.104ZDEBGnexus (service-firewall-rule-progator): resolved 3 rules for sleds background_task = service_firewall_rule_propagation
107192025-08-26T21:56:02.121ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
107202025-08-26T21:56:02.121ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 434.677919ms iteration = 11 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
107212025-08-26T21:56:02.180ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
107222025-08-26T21:56:02.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
107232025-08-26T21:56:02.183ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 518.748244ms iteration = 21 status = null
107242025-08-26T21:56:02.183ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
107252025-08-26T21:56:02.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
107262025-08-26T21:56:02.185ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107272025-08-26T21:56:02.187ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
107282025-08-26T21:56:02.187ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
107292025-08-26T21:56:02.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
107302025-08-26T21:56:02.189ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
107312025-08-26T21:56:02.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
107322025-08-26T21:56:02.191ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107332025-08-26T21:56:02.191ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, time_created: 2025-08-26T21:46:02.113579Z, time_modified: 2025-08-26T21:46:02.113579Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
107342025-08-26T21:56:02.191ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
107352025-08-26T21:56:02.193ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
107362025-08-26T21:56:02.193ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
107372025-08-26T21:56:02.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
107382025-08-26T21:56:02.220ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
107392025-08-26T21:56:02.220ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "3e434cb7-d45c-4019-96cf-e94885d054c5", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
107402025-08-26T21:56:02.220ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
107412025-08-26T21:56:02.222ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "a4e895ac-4fc7-4302-b639-40a5dd1a1ea6", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
107422025-08-26T21:56:02.222ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
107432025-08-26T21:56:02.224ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
107442025-08-26T21:56:02.224ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 560.045304ms iteration = 13 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
107452025-08-26T21:56:02.225ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("5362db44-dfc6-41c1-8ec8-32ab3e86b7b3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("5e2b44ef-e7e6-47ec-9087-a2fb44127ede.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("70604151-9a25-4539-bcae-b8d33d91f6a5.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("be83bf79-a7c5-40a7-be80-31ec2f206604.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("d35c8171-47d8-4704-93d7-f07739a9281c.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d35c8171-47d8-4704-93d7-f07739a9281c.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::7)) }], valid_until: Instant { tv_sec: 1466, tv_nsec: 84013619 } })
107462025-08-26T21:56:02.226ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
107472025-08-26T21:56:02.226ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
107482025-08-26T21:56:02.226ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
107492025-08-26T21:56:02.226ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "16be47e2-cb72-4a26-a5d0-fcb18d94ae41", "content-length": "476", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
107502025-08-26T21:56:02.226ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
107512025-08-26T21:56:02.226ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f21b512-adbe-4e12-b9ec-9dd27a579872", "content-length": "734", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} })
107522025-08-26T21:56:02.227ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
107532025-08-26T21:56:02.227ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8746907c-333c-47ab-be50-f9216024102e", "content-length": "111", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
107542025-08-26T21:56:02.227ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
107552025-08-26T21:56:02.227ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "98a856f2-1fcb-475d-b138-7c31b58a442c", "content-length": "111", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
107562025-08-26T21:56:02.227ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
107572025-08-26T21:56:02.228ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "290211a5-9765-44d5-8a42-484df7082372", "content-length": "163", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} })
107582025-08-26T21:56:02.228ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
107592025-08-26T21:56:02.228ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a8a6ae4-ae29-409b-acb1-32d01af8e3a3", "content-length": "163", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} })
107602025-08-26T21:56:02.228ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
107612025-08-26T21:56:02.229ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0425c964-6182-4076-be3b-0fe9216690ac", "content-length": "166", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} })
107622025-08-26T21:56:02.229ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
107632025-08-26T21:56:02.229ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1584eb12-e56a-447e-8316-3fe60d73a80a", "content-length": "166", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
107642025-08-26T21:56:02.229ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
107652025-08-26T21:56:02.229ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4e753d6-30a0-4d20-a31c-ae1a58ff9458", "content-length": "702", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
107662025-08-26T21:56:02.229ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
107672025-08-26T21:56:02.229ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7ed25c6-6ab4-4391-9163-1e73b5f5ee44", "content-length": "727", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
107682025-08-26T21:56:02.229ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
107692025-08-26T21:56:02.230ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "012c6fe8-972c-48b9-94bd-f744a5a381c7", "content-length": "729", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
107702025-08-26T21:56:02.230ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
107712025-08-26T21:56:02.230ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ff18783-980e-4574-9325-65bcf1061f3d", "content-length": "728", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
107722025-08-26T21:56:02.230ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
107732025-08-26T21:56:02.230ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd0347ea-8aab-4ff8-a0fb-7cf6f5e0357b", "content-length": "722", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
107742025-08-26T21:56:02.230ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
107752025-08-26T21:56:02.230ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6ea22ce-eccb-4c25-95e8-2c17f8493dbb", "content-length": "10", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
107762025-08-26T21:56:02.230ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
107772025-08-26T21:56:02.230ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a54ce58-32a5-4472-a25d-db1ab66584b7", "content-length": "144", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
107782025-08-26T21:56:02.230ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
107792025-08-26T21:56:02.231ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1dd5e43-1e71-41d7-8073-65270f9c25b6", "content-length": "147", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
107802025-08-26T21:56:02.231ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
107812025-08-26T21:56:02.231ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e92d1ee2-a24c-4349-b1c8-1545f43e9865", "content-length": "109", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
107822025-08-26T21:56:02.231ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
107832025-08-26T21:56:02.231ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8423151c-2174-49f5-b87b-10de27bd70d3", "content-length": "109", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} })
107842025-08-26T21:56:02.231ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
107852025-08-26T21:56:02.232ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "72ebc88d-73d7-434d-abd8-0e866fe8a53c", "content-length": "163", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
107862025-08-26T21:56:02.232ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
107872025-08-26T21:56:02.232ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "eccd8ccd-28b3-4c7f-bffa-c65ae4e7a0cd", "content-length": "163", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} })
107882025-08-26T21:56:02.232ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
107892025-08-26T21:56:02.232ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3dac98de-7f5b-4fcb-b28e-f0fe698f8fbe", "content-length": "166", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
107902025-08-26T21:56:02.233ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
107912025-08-26T21:56:02.233ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e5dddd2-7247-405b-90b2-14ca042ae1cf", "content-length": "166", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
107922025-08-26T21:56:02.233ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
107932025-08-26T21:56:02.233ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d466b6cc-d784-4f28-9c37-0a09c7c2d136", "content-length": "702", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
107942025-08-26T21:56:02.233ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
107952025-08-26T21:56:02.233ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2bba047-6fe9-4385-b92c-9b59fa191cbe", "content-length": "727", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
107962025-08-26T21:56:02.233ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
107972025-08-26T21:56:02.233ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "47809ca7-82f9-43ba-9051-7d92cfa35e66", "content-length": "729", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
107982025-08-26T21:56:02.233ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
107992025-08-26T21:56:02.234ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a35b23a-e8e5-4bc5-9002-90fb32e5d11d", "content-length": "728", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
108002025-08-26T21:56:02.234ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
108012025-08-26T21:56:02.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
108022025-08-26T21:56:02.236ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108032025-08-26T21:56:02.238ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
108042025-08-26T21:56:02.238ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
108052025-08-26T21:56:02.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
108062025-08-26T21:56:02.240ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108072025-08-26T21:56:02.242ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
108082025-08-26T21:56:02.242ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
108092025-08-26T21:56:02.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
108102025-08-26T21:56:02.248ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
108112025-08-26T21:56:02.248ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
108122025-08-26T21:56:02.249ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "038e95a5-9808-4bd4-bd26-079580ffe44c", "content-length": "372", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
108132025-08-26T21:56:02.249ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
108142025-08-26T21:56:02.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
108152025-08-26T21:56:02.253ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = tuf_artifact_replication body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/artifacts-config
108162025-08-26T21:56:02.254ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts-config", status: 204, headers: {"x-request-id": "0db32069-0721-43be-b0ee-189f237ac44b", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
108172025-08-26T21:56:02.254ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = tuf_artifact_replication body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/artifacts
108182025-08-26T21:56:02.254ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "69171035-9732-4a0a-a03d-ca713bdfeaa3", "content-length": "26", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
108192025-08-26T21:56:02.254ZINFOnexus (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication file = nexus/src/app/background/tasks/tuf_artifact_replication.rs:672 sled = http://[fd00:1122:3344:101::1]:12345
108202025-08-26T21:56:02.256ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 569.60511ms iteration = 3 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":2,"put_config_err":0,"put_config_ok":2,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"generation":1,"operation":"put_config","target_sled":"dd0c655a-95c2-48fc-b4c3-9733ed4b0faa","time":"2025-08-26T21:51:02.015071525Z"},{"operation":"list","target_sled":"dd0c655a-95c2-48fc-b4c3-9733ed4b0faa","time":"2025-08-26T21:51:02.020355155Z"},{"generation":1,"operation":"put_config","target_sled":"dd0c655a-95c2-48fc-b4c3-9733ed4b0faa","time":"2025-08-26T21:56:02.254385217Z"},{"operation":"list","target_sled":"dd0c655a-95c2-48fc-b4c3-9733ed4b0faa","time":"2025-08-26T21:56:02.254568842Z"}]}
108212025-08-26T21:56:02.267ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
108222025-08-26T21:56:02.267ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
108232025-08-26T21:56:02.267ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "777983fb-2e9c-4e4d-ab02-2466409b03d0", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
108242025-08-26T21:56:02.267ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
108252025-08-26T21:56:02.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
108262025-08-26T21:56:02.269ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
108272025-08-26T21:56:02.269ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a321c99-392c-40ab-b005-78d495cbfef4", "content-length": "691", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
108282025-08-26T21:56:02.269ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
108292025-08-26T21:56:02.269ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
108302025-08-26T21:56:02.269ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
108312025-08-26T21:56:02.270ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
108322025-08-26T21:56:02.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
108332025-08-26T21:56:02.271ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 597.59349ms iteration = 21 status = {}
108342025-08-26T21:56:02.272ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 607.996315ms iteration = 21 status = {}
108352025-08-26T21:56:02.274ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
108362025-08-26T21:56:02.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
108372025-08-26T21:56:02.278ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
108382025-08-26T21:56:02.278ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
108392025-08-26T21:56:02.278ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "bed64f8d-e3c3-4cd7-9ce2-e23c80a8e7c2", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
108402025-08-26T21:56:02.280ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 593.243643ms iteration = 21 status = {}
108412025-08-26T21:56:02.360ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
108422025-08-26T21:56:02.429ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
108432025-08-26T21:56:02.429ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
108442025-08-26T21:56:02.435ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
108452025-08-26T21:56:02.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
108462025-08-26T21:56:02.439ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
108472025-08-26T21:56:02.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
108482025-08-26T21:56:02.441ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108492025-08-26T21:56:02.444ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
108502025-08-26T21:56:02.444ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
108512025-08-26T21:56:02.445ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "d6c5d48f-1a5d-4fd9-8a76-afd19236e55a", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
108522025-08-26T21:56:02.447ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:491
108532025-08-26T21:56:02.447ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 761.077876ms
108542025-08-26T21:56:02.447ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 761.095876ms iteration = 3 status = {}
108552025-08-26T21:56:02.455ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
108562025-08-26T21:56:02.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
108572025-08-26T21:56:02.457ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
108582025-08-26T21:56:02.457ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
108592025-08-26T21:56:02.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
108602025-08-26T21:56:02.469ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
108612025-08-26T21:56:02.469ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
108622025-08-26T21:56:02.469ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
108632025-08-26T21:56:02.469ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
108642025-08-26T21:56:02.469ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
108652025-08-26T21:56:02.469ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
108662025-08-26T21:56:02.469ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
108672025-08-26T21:56:02.469ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
108682025-08-26T21:56:02.469ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
108692025-08-26T21:56:02.469ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
108702025-08-26T21:56:02.470ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
108712025-08-26T21:56:02.470ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
108722025-08-26T21:56:02.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
108732025-08-26T21:56:02.470ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
108742025-08-26T21:56:02.471ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "52d43e59-3d18-416f-bec4-ebfa0d773c9f", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
108752025-08-26T21:56:02.471ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
108762025-08-26T21:56:02.473ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "522de9d1-febe-4077-a3a8-cbabb14c50f7", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
108772025-08-26T21:56:02.473ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
108782025-08-26T21:56:02.473ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
108792025-08-26T21:56:02.473ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
108802025-08-26T21:56:02.475ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "88706b60-bd7b-4c1a-8458-846d8a4ae935", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
108812025-08-26T21:56:02.475ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
108822025-08-26T21:56:02.476ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
108832025-08-26T21:56:02.477ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 812.28584ms iteration = 13 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
108842025-08-26T21:56:02.483ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "fdbb59e9-dbda-43af-a9c2-d8bd12d3b536", "content-length": "88498", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
108852025-08-26T21:56:02.530ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
108862025-08-26T21:56:02.530ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
108872025-08-26T21:56:02.530ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
108882025-08-26T21:56:02.530ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
108892025-08-26T21:56:02.530ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
108902025-08-26T21:56:02.530ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
108912025-08-26T21:56:02.530ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
108922025-08-26T21:56:02.531ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
108932025-08-26T21:56:02.531ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "62f63aa0-faa5-462e-8457-279ec04e4732", "content-length": "64", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} })
108942025-08-26T21:56:02.531ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
108952025-08-26T21:56:02.531ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "645075d5-650a-4420-8be6-c7c5d4c82b3a", "content-length": "64", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} })
108962025-08-26T21:56:02.531ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
108972025-08-26T21:56:02.531ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "11f8967a-32dd-4c8f-aa23-bbd5ba6ef0b2", "content-length": "64", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} })
108982025-08-26T21:56:02.531ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
108992025-08-26T21:56:02.531ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "80f52737-4696-4af0-bfce-e8246e24fedc", "content-length": "64", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} })
109002025-08-26T21:56:02.531ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
109012025-08-26T21:56:02.531ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c648148f-4433-4a2a-9b86-8b37e230b568", "content-length": "64", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} })
109022025-08-26T21:56:02.531ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
109032025-08-26T21:56:02.572ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1dba07c-c601-4d8a-8740-9ccc13668485", "content-length": "389023", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} })
109042025-08-26T21:56:02.573ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "08f8302e-763a-467a-9698-0ae75eddfbf9", "content-length": "396582", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} })
109052025-08-26T21:56:02.575ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "cfe1f525-7e11-4216-998a-22e45bc0764d", "content-length": "402319", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} })
109062025-08-26T21:56:02.576ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "f30c07db-0cc6-439b-8fe6-a423950746a8", "content-length": "387396", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} })
109072025-08-26T21:56:02.580ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "07893ab9-f4ef-4e79-98b4-6573c42b91ed", "content-length": "411751", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} })
109082025-08-26T21:56:02.583ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
109092025-08-26T21:56:02.583ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
109102025-08-26T21:56:02.583ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
109112025-08-26T21:56:02.583ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
109122025-08-26T21:56:02.583ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
109132025-08-26T21:56:02.597ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "1c79c739-6aea-4e8e-a5f7-60614fcf4113", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
109142025-08-26T21:56:02.597ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
109152025-08-26T21:56:02.597ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
109162025-08-26T21:56:02.598ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3b097ba-8c72-4d93-856d-5ebe948ed628", "content-length": "682", "date": "Tue, 26 Aug 2025 21:56:01 GMT"} })
109172025-08-26T21:56:02.598ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
109182025-08-26T21:56:02.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
109192025-08-26T21:56:02.602ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
109202025-08-26T21:56:02.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
109212025-08-26T21:56:02.606ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
109222025-08-26T21:56:02.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
109232025-08-26T21:56:02.608ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
109242025-08-26T21:56:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
109252025-08-26T21:56:02.617ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 943.085679ms iteration = 21 status = {}
109262025-08-26T21:56:02.624ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = e17ff130-2852-42b2-9cfd-58a3a8b33eb6 (service)
109272025-08-26T21:56:02.624ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
109282025-08-26T21:56:02.627ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e9a19a6-e70a-4b41-86ae-5907e4244f0c", "content-length": "128", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} })
109292025-08-26T21:56:02.629ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
109302025-08-26T21:56:02.733ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 852fd1a3-de9a-47e8-a667-5e8dab880ad3 (service)
109312025-08-26T21:56:02.733ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
109322025-08-26T21:56:02.734ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5b3928d-a7b8-4432-9a26-18cb2a68af8f", "content-length": "9110", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} })
109332025-08-26T21:56:02.740ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 852fd1a3-de9a-47e8-a667-5e8dab880ad3 (service)
109342025-08-26T21:56:02.741ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = c68f1501-294c-4874-92ca-8fc0546a5870 (service)
109352025-08-26T21:56:02.741ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
109362025-08-26T21:56:02.742ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f1c15b1-d65c-48b4-b05c-affd3539b9f7", "content-length": "9110", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} })
109372025-08-26T21:56:02.744ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = c68f1501-294c-4874-92ca-8fc0546a5870 (service)
109382025-08-26T21:56:02.746ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = d741a72f-8745-4723-9e80-e1e72987266b (service)
109392025-08-26T21:56:02.746ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
109402025-08-26T21:56:02.747ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "94b0610a-a8bd-455b-83a8-581196b4f2f0", "content-length": "9110", "date": "Tue, 26 Aug 2025 21:56:02 GMT"} })
109412025-08-26T21:56:02.747ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = d741a72f-8745-4723-9e80-e1e72987266b (service)
109422025-08-26T21:56:02.748ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
109432025-08-26T21:56:02.748ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
109442025-08-26T21:56:02.749ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
109452025-08-26T21:56:02.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
109462025-08-26T21:56:02.752ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109472025-08-26T21:56:02.754ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
109482025-08-26T21:56:02.760ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
109492025-08-26T21:56:02.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
109502025-08-26T21:56:03.065ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 28fbc916-7029-4e55-95e8-37ed39eb5e3f file = nexus/db-queries/src/db/datastore/inventory.rs:1614
109512025-08-26T21:56:03.065ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 28fbc916-7029-4e55-95e8-37ed39eb5e3f time_started = 2025-08-26 21:56:02.226016 UTC
109522025-08-26T21:56:03.066ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 23 reason = Dependency
109532025-08-26T21:56:03.066ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 14 reason = Dependency
109542025-08-26T21:56:03.066ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
109552025-08-26T21:56:03.066ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 9.45\u{b5}s iteration = 14 status = "Disabled"
109562025-08-26T21:56:03.066ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 5 reason = Dependency
109572025-08-26T21:56:03.066ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
109582025-08-26T21:56:03.066ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
109592025-08-26T21:56:03.066ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.400119132s iteration = 2 status = {"collection_id":"28fbc916-7029-4e55-95e8-37ed39eb5e3f","time_done":"2025-08-26 21:56:02.748374 UTC","time_started":"2025-08-26 21:56:02.226016 UTC"}
109602025-08-26T21:56:03.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
109612025-08-26T21:56:03.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
109622025-08-26T21:56:03.068ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109632025-08-26T21:56:03.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109642025-08-26T21:56:03.072ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
109652025-08-26T21:56:03.072ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
109662025-08-26T21:56:03.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
109672025-08-26T21:56:03.074ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
109682025-08-26T21:56:03.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
109692025-08-26T21:56:03.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
109702025-08-26T21:56:03.076ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
109712025-08-26T21:56:03.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
109722025-08-26T21:56:03.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
109732025-08-26T21:56:03.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
109742025-08-26T21:56:03.079ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109752025-08-26T21:56:03.082ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
109762025-08-26T21:56:03.082ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
109772025-08-26T21:56:03.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
109782025-08-26T21:56:03.095ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 29.505375ms iteration = 23 status = {"physical_disks_added":0}
109792025-08-26T21:56:03.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
109802025-08-26T21:56:03.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
109812025-08-26T21:56:03.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109822025-08-26T21:56:03.143ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
109832025-08-26T21:56:03.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
109842025-08-26T21:56:03.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
109852025-08-26T21:56:03.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109862025-08-26T21:56:03.148ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
109872025-08-26T21:56:03.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
109882025-08-26T21:56:03.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
109892025-08-26T21:56:03.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
109902025-08-26T21:56:03.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
109912025-08-26T21:56:03.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109922025-08-26T21:56:03.155ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
109932025-08-26T21:56:03.155ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
109942025-08-26T21:56:03.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
109952025-08-26T21:56:03.157ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109962025-08-26T21:56:03.160ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
109972025-08-26T21:56:03.161ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
109982025-08-26T21:56:03.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
109992025-08-26T21:56:03.164ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:113 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
110002025-08-26T21:56:03.164ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 98.943441ms iteration = 5 status = {"blueprint_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","inventory_collection_id":"28fbc916-7029-4e55-95e8-37ed39eb5e3f","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
110012025-08-26T21:56:05.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 360 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = a58524e9-bf18-4260-bcb9-2e9dcc3c5e73 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
110022025-08-26T21:56:06.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 122 reason = Timeout
110032025-08-26T21:56:06.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
110042025-08-26T21:56:06.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
110052025-08-26T21:56:06.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110062025-08-26T21:56:06.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
110072025-08-26T21:56:06.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
110082025-08-26T21:56:06.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
110092025-08-26T21:56:06.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
110102025-08-26T21:56:06.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.608472ms iteration = 122 status = {"chicken_switches_updated":false}
110112025-08-26T21:56:11.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 123 reason = Timeout
110122025-08-26T21:56:11.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
110132025-08-26T21:56:11.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
110142025-08-26T21:56:11.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110152025-08-26T21:56:11.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
110162025-08-26T21:56:11.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
110172025-08-26T21:56:11.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
110182025-08-26T21:56:11.672ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
110192025-08-26T21:56:11.672ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.19159ms iteration = 123 status = {"chicken_switches_updated":false}
110202025-08-26T21:56:11.673ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 62 reason = Timeout
110212025-08-26T21:56:11.673ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
110222025-08-26T21:56:11.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
110232025-08-26T21:56:11.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110242025-08-26T21:56:11.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
110252025-08-26T21:56:11.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
110262025-08-26T21:56:11.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
110272025-08-26T21:56:11.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
110282025-08-26T21:56:11.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
110292025-08-26T21:56:11.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110302025-08-26T21:56:11.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
110312025-08-26T21:56:11.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
110322025-08-26T21:56:11.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
110332025-08-26T21:56:11.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 62 reason = Timeout
110342025-08-26T21:56:11.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
110352025-08-26T21:56:11.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
110362025-08-26T21:56:11.703ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
110372025-08-26T21:56:11.703ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.184778ms iteration = 62 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
110382025-08-26T21:56:11.722ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
110392025-08-26T21:56:11.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
110402025-08-26T21:56:11.724ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 38.430617ms iteration = 62 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
110412025-08-26T21:56:15.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 374 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 782b9696-e82e-4846-ba09-499e9dc21019 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
110422025-08-26T21:56:16.665ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 124 reason = Timeout
110432025-08-26T21:56:16.665ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
110442025-08-26T21:56:16.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
110452025-08-26T21:56:16.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110462025-08-26T21:56:16.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
110472025-08-26T21:56:16.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
110482025-08-26T21:56:16.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
110492025-08-26T21:56:16.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
110502025-08-26T21:56:16.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.202592ms iteration = 124 status = {"chicken_switches_updated":false}
110512025-08-26T21:56:21.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 125 reason = Timeout
110522025-08-26T21:56:21.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
110532025-08-26T21:56:21.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
110542025-08-26T21:56:21.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110552025-08-26T21:56:21.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
110562025-08-26T21:56:21.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
110572025-08-26T21:56:21.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
110582025-08-26T21:56:21.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
110592025-08-26T21:56:21.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.011978ms iteration = 125 status = {"chicken_switches_updated":false}
110602025-08-26T21:56:21.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 63 reason = Timeout
110612025-08-26T21:56:21.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
110622025-08-26T21:56:21.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
110632025-08-26T21:56:21.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110642025-08-26T21:56:21.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
110652025-08-26T21:56:21.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
110662025-08-26T21:56:21.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
110672025-08-26T21:56:21.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
110682025-08-26T21:56:21.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
110692025-08-26T21:56:21.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110702025-08-26T21:56:21.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
110712025-08-26T21:56:21.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
110722025-08-26T21:56:21.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
110732025-08-26T21:56:21.685ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 63 reason = Timeout
110742025-08-26T21:56:21.685ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
110752025-08-26T21:56:21.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
110762025-08-26T21:56:21.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
110772025-08-26T21:56:21.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
110782025-08-26T21:56:21.689ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.696113ms iteration = 63 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
110792025-08-26T21:56:21.698ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
110802025-08-26T21:56:21.698ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.268627ms iteration = 63 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
110812025-08-26T21:56:25.351ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 376 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = e7839496-1fdb-47e8-b64b-34f4d5b9da76 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
110822025-08-26T21:56:26.665ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 126 reason = Timeout
110832025-08-26T21:56:26.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
110842025-08-26T21:56:26.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
110852025-08-26T21:56:26.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110862025-08-26T21:56:26.670ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
110872025-08-26T21:56:26.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
110882025-08-26T21:56:26.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
110892025-08-26T21:56:26.672ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
110902025-08-26T21:56:26.672ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.687641ms iteration = 126 status = {"chicken_switches_updated":false}
110912025-08-26T21:56:31.664ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 22 reason = Timeout
110922025-08-26T21:56:31.664ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 22 reason = Timeout
110932025-08-26T21:56:31.664ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 22 reason = Timeout
110942025-08-26T21:56:31.664ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
110952025-08-26T21:56:31.664ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
110962025-08-26T21:56:31.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
110972025-08-26T21:56:31.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
110982025-08-26T21:56:31.665ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 127 reason = Timeout
110992025-08-26T21:56:31.665ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
111002025-08-26T21:56:31.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
111012025-08-26T21:56:31.666ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1495, tv_nsec: 525754135 } })
111022025-08-26T21:56:31.667ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1495, tv_nsec: 526002860 } })
111032025-08-26T21:56:31.667ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
111042025-08-26T21:56:31.667ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.899545ms iteration = 22 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
111052025-08-26T21:56:31.667ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
111062025-08-26T21:56:31.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111072025-08-26T21:56:31.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
111082025-08-26T21:56:31.672ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
111092025-08-26T21:56:31.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
111102025-08-26T21:56:31.673ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 22 reason = Timeout
111112025-08-26T21:56:31.673ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
111122025-08-26T21:56:31.673ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 24 reason = Timeout
111132025-08-26T21:56:31.673ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
111142025-08-26T21:56:31.673ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 22 reason = Timeout
111152025-08-26T21:56:31.673ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 22 reason = Timeout
111162025-08-26T21:56:31.673ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
111172025-08-26T21:56:31.673ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
111182025-08-26T21:56:31.673ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 22 reason = Timeout
111192025-08-26T21:56:31.673ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
111202025-08-26T21:56:31.673ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 64 reason = Timeout
111212025-08-26T21:56:31.673ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
111222025-08-26T21:56:31.673ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 22 reason = Timeout
111232025-08-26T21:56:31.673ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
111242025-08-26T21:56:31.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
111252025-08-26T21:56:31.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.196725ms iteration = 127 status = {"chicken_switches_updated":false}
111262025-08-26T21:56:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
111272025-08-26T21:56:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
111282025-08-26T21:56:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
111292025-08-26T21:56:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
111302025-08-26T21:56:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
111312025-08-26T21:56:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
111322025-08-26T21:56:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
111332025-08-26T21:56:31.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111342025-08-26T21:56:31.675ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111352025-08-26T21:56:31.675ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111362025-08-26T21:56:31.676ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111372025-08-26T21:56:31.676ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111382025-08-26T21:56:31.678ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
111392025-08-26T21:56:31.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
111402025-08-26T21:56:31.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
111412025-08-26T21:56:31.679ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
111422025-08-26T21:56:31.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
111432025-08-26T21:56:31.679ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
111442025-08-26T21:56:31.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
111452025-08-26T21:56:31.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
111462025-08-26T21:56:31.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
111472025-08-26T21:56:31.679ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
111482025-08-26T21:56:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
111492025-08-26T21:56:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
111502025-08-26T21:56:31.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
111512025-08-26T21:56:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
111522025-08-26T21:56:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
111532025-08-26T21:56:31.681ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
111542025-08-26T21:56:31.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
111552025-08-26T21:56:31.682ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
111562025-08-26T21:56:31.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
111572025-08-26T21:56:31.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
111582025-08-26T21:56:31.682ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1495, tv_nsec: 541770246 } })
111592025-08-26T21:56:31.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
111602025-08-26T21:56:31.683ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
111612025-08-26T21:56:31.683ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
111622025-08-26T21:56:31.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
111632025-08-26T21:56:31.684ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111642025-08-26T21:56:31.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111652025-08-26T21:56:31.686ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 22 reason = Timeout
111662025-08-26T21:56:31.686ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 22 reason = Timeout
111672025-08-26T21:56:31.686ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
111682025-08-26T21:56:31.686ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
111692025-08-26T21:56:31.686ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
111702025-08-26T21:56:31.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 64 reason = Timeout
111712025-08-26T21:56:31.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
111722025-08-26T21:56:31.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
111732025-08-26T21:56:31.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
111742025-08-26T21:56:31.686ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 22 reason = Timeout
111752025-08-26T21:56:31.686ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 22 reason = Timeout
111762025-08-26T21:56:31.686ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
111772025-08-26T21:56:31.686ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 22 reason = Timeout
111782025-08-26T21:56:31.686ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 12.9\u{b5}s iteration = 22 status = {"disabled":true,"errors":[],"sps":[]}
111792025-08-26T21:56:31.686ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
111802025-08-26T21:56:31.686ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 22 reason = Timeout
111812025-08-26T21:56:31.686ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 22 reason = Timeout
111822025-08-26T21:56:31.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
111832025-08-26T21:56:31.686ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
111842025-08-26T21:56:31.686ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
111852025-08-26T21:56:31.686ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
111862025-08-26T21:56:31.686ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 22 reason = Timeout
111872025-08-26T21:56:31.686ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
111882025-08-26T21:56:31.686ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
111892025-08-26T21:56:31.686ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 22 reason = Timeout
111902025-08-26T21:56:31.686ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
111912025-08-26T21:56:31.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
111922025-08-26T21:56:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
111932025-08-26T21:56:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
111942025-08-26T21:56:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
111952025-08-26T21:56:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
111962025-08-26T21:56:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
111972025-08-26T21:56:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
111982025-08-26T21:56:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
111992025-08-26T21:56:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
112002025-08-26T21:56:31.688ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
112012025-08-26T21:56:31.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
112022025-08-26T21:56:31.688ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112032025-08-26T21:56:31.688ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112042025-08-26T21:56:31.689ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 2.749692ms iteration = 22 status = {"errors":[],"garbage_collect_requested":[]}
112052025-08-26T21:56:31.691ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
112062025-08-26T21:56:31.691ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
112072025-08-26T21:56:31.692ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
112082025-08-26T21:56:31.692ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
112092025-08-26T21:56:31.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
112102025-08-26T21:56:31.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
112112025-08-26T21:56:31.694ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 7.478998ms iteration = 22 status = {"errors":[],"requests_created_ok":[]}
112122025-08-26T21:56:31.736ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
112132025-08-26T21:56:31.736ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 62.56122ms iteration = 64 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
112142025-08-26T21:56:31.746ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
112152025-08-26T21:56:31.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
112162025-08-26T21:56:31.747ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 73.867985ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
112172025-08-26T21:56:31.748ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
112182025-08-26T21:56:31.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
112192025-08-26T21:56:31.750ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112202025-08-26T21:56:31.754ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
112212025-08-26T21:56:31.754ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
112222025-08-26T21:56:31.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
112232025-08-26T21:56:31.754ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
112242025-08-26T21:56:31.754ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
112252025-08-26T21:56:31.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
112262025-08-26T21:56:31.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
112272025-08-26T21:56:31.756ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112282025-08-26T21:56:31.756ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 69.806643ms iteration = 64 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
112292025-08-26T21:56:31.756ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 82.740156ms iteration = 22 status = 0
112302025-08-26T21:56:31.756ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
112312025-08-26T21:56:31.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
112322025-08-26T21:56:31.757ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112332025-08-26T21:56:31.760ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
112342025-08-26T21:56:31.760ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
112352025-08-26T21:56:31.760ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
112362025-08-26T21:56:31.760ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
112372025-08-26T21:56:31.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
112382025-08-26T21:56:31.760ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
112392025-08-26T21:56:31.760ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
112402025-08-26T21:56:31.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
112412025-08-26T21:56:31.761ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1d88daf-070b-4b8c-a749-117bffe10bf5", "content-length": "26", "date": "Tue, 26 Aug 2025 21:56:31 GMT"} })
112422025-08-26T21:56:31.761ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
112432025-08-26T21:56:31.763ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
112442025-08-26T21:56:31.763ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 76.717269ms iteration = 22 status = {"errors":[],"finish_invoked_ok":[]}
112452025-08-26T21:56:31.764ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
112462025-08-26T21:56:31.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
112472025-08-26T21:56:31.764ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
112482025-08-26T21:56:31.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
112492025-08-26T21:56:31.765ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
112502025-08-26T21:56:31.765ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
112512025-08-26T21:56:31.765ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
112522025-08-26T21:56:31.765ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 79.018332ms iteration = 22 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
112532025-08-26T21:56:31.765ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "33b2a16c-0cff-455f-8e04-ca4447ee4c3e", "content-length": "26", "date": "Tue, 26 Aug 2025 21:56:31 GMT"} })
112542025-08-26T21:56:31.765ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
112552025-08-26T21:56:31.767ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
112562025-08-26T21:56:31.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
112572025-08-26T21:56:31.767ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
112582025-08-26T21:56:31.769ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112592025-08-26T21:56:31.771ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
112602025-08-26T21:56:31.771ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
112612025-08-26T21:56:31.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
112622025-08-26T21:56:31.773ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112632025-08-26T21:56:31.777ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
112642025-08-26T21:56:31.777ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
112652025-08-26T21:56:31.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
112662025-08-26T21:56:31.781ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
112672025-08-26T21:56:31.781ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
112682025-08-26T21:56:31.781ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
112692025-08-26T21:56:31.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
112702025-08-26T21:56:31.815ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
112712025-08-26T21:56:31.816ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
112722025-08-26T21:56:31.816ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bb1a87f-a023-45a1-a62b-bbb5bb72c129", "content-length": "26", "date": "Tue, 26 Aug 2025 21:56:31 GMT"} })
112732025-08-26T21:56:31.816ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
112742025-08-26T21:56:31.818ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
112752025-08-26T21:56:31.820ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
112762025-08-26T21:56:31.820ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 146.158575ms iteration = 22 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
112772025-08-26T21:56:31.822ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 135.507035ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
112782025-08-26T21:56:31.827ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
112792025-08-26T21:56:31.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
112802025-08-26T21:56:31.836ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
112812025-08-26T21:56:31.836ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
112822025-08-26T21:56:31.836ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
112832025-08-26T21:56:31.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
112842025-08-26T21:56:31.849ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
112852025-08-26T21:56:31.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
112862025-08-26T21:56:31.858ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 193.933572ms iteration = 22 status = null
112872025-08-26T21:56:31.861ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 197.531483ms iteration = 22 status = {}
112882025-08-26T21:56:31.875ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 201.853841ms iteration = 24 status = {"physical_disks_added":0}
112892025-08-26T21:56:31.875ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 189.069562ms iteration = 22 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
112902025-08-26T21:56:31.889ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
112912025-08-26T21:56:31.889ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
112922025-08-26T21:56:31.889ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 203.384535ms iteration = 22 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
112932025-08-26T21:56:31.928ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
112942025-08-26T21:56:31.928ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
112952025-08-26T21:56:31.929ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd601cf0-3141-4587-9804-060a2d8addeb", "content-length": "372", "date": "Tue, 26 Aug 2025 21:56:31 GMT"} })
112962025-08-26T21:56:31.929ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
112972025-08-26T21:56:31.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
112982025-08-26T21:56:31.947ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
112992025-08-26T21:56:31.947ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "17b5f0f4-4529-4025-b6c8-66fac2a68145", "content-length": "691", "date": "Tue, 26 Aug 2025 21:56:31 GMT"} })
113002025-08-26T21:56:31.947ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
113012025-08-26T21:56:31.947ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
113022025-08-26T21:56:31.947ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
113032025-08-26T21:56:31.949ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 275.321906ms iteration = 22 status = {}
113042025-08-26T21:56:31.976ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
113052025-08-26T21:56:31.976ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
113062025-08-26T21:56:31.976ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "5cc568ed-9d54-47c2-9e88-908cde3b8101", "date": "Tue, 26 Aug 2025 21:56:31 GMT"} })
113072025-08-26T21:56:31.976ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
113082025-08-26T21:56:31.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
113092025-08-26T21:56:31.978ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
113102025-08-26T21:56:31.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
113112025-08-26T21:56:31.981ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
113122025-08-26T21:56:31.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
113132025-08-26T21:56:31.984ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
113142025-08-26T21:56:31.984ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
113152025-08-26T21:56:31.984ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "5df637e4-1c12-4d16-abca-e7dbb9890bee", "date": "Tue, 26 Aug 2025 21:56:31 GMT"} })
113162025-08-26T21:56:31.986ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 299.219955ms iteration = 22 status = {}
113172025-08-26T21:56:31.991ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
113182025-08-26T21:56:31.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
113192025-08-26T21:56:31.993ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
113202025-08-26T21:56:31.993ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
113212025-08-26T21:56:31.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
113222025-08-26T21:56:32.005ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
113232025-08-26T21:56:32.005ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
113242025-08-26T21:56:32.006ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
113252025-08-26T21:56:32.006ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
113262025-08-26T21:56:32.006ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
113272025-08-26T21:56:32.006ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
113282025-08-26T21:56:32.006ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
113292025-08-26T21:56:32.006ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
113302025-08-26T21:56:32.006ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
113312025-08-26T21:56:32.006ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
113322025-08-26T21:56:32.006ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
113332025-08-26T21:56:32.006ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
113342025-08-26T21:56:32.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
113352025-08-26T21:56:32.008ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
113362025-08-26T21:56:32.008ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
113372025-08-26T21:56:32.127ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "abc298b5-c178-4968-bcfc-0bc5afcfa985", "date": "Tue, 26 Aug 2025 21:56:31 GMT"} })
113382025-08-26T21:56:32.127ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
113392025-08-26T21:56:32.127ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
113402025-08-26T21:56:32.133ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "6407727c-1574-4615-88d8-49888b48ffbc", "content-length": "682", "date": "Tue, 26 Aug 2025 21:56:32 GMT"} })
113412025-08-26T21:56:32.133ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
113422025-08-26T21:56:32.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
113432025-08-26T21:56:32.137ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
113442025-08-26T21:56:32.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
113452025-08-26T21:56:32.141ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
113462025-08-26T21:56:32.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
113472025-08-26T21:56:32.142ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
113482025-08-26T21:56:32.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
113492025-08-26T21:56:32.150ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 476.290906ms iteration = 22 status = {}
113502025-08-26T21:56:35.351ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 340 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 654d5ec5-c1f1-4df3-852e-9b78dd4cf75f response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
113512025-08-26T21:56:36.665ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 128 reason = Timeout
113522025-08-26T21:56:36.665ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
113532025-08-26T21:56:36.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
113542025-08-26T21:56:36.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113552025-08-26T21:56:36.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
113562025-08-26T21:56:36.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
113572025-08-26T21:56:36.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
113582025-08-26T21:56:36.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
113592025-08-26T21:56:36.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.190804ms iteration = 128 status = {"chicken_switches_updated":false}
113602025-08-26T21:56:41.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 129 reason = Timeout
113612025-08-26T21:56:41.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
113622025-08-26T21:56:41.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
113632025-08-26T21:56:41.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113642025-08-26T21:56:41.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
113652025-08-26T21:56:41.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
113662025-08-26T21:56:41.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
113672025-08-26T21:56:41.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
113682025-08-26T21:56:41.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.385146ms iteration = 129 status = {"chicken_switches_updated":false}
113692025-08-26T21:56:41.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 65 reason = Timeout
113702025-08-26T21:56:41.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
113712025-08-26T21:56:41.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
113722025-08-26T21:56:41.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113732025-08-26T21:56:41.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
113742025-08-26T21:56:41.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
113752025-08-26T21:56:41.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
113762025-08-26T21:56:41.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
113772025-08-26T21:56:41.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
113782025-08-26T21:56:41.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113792025-08-26T21:56:41.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
113802025-08-26T21:56:41.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
113812025-08-26T21:56:41.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
113822025-08-26T21:56:41.685ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 65 reason = Timeout
113832025-08-26T21:56:41.685ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
113842025-08-26T21:56:41.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
113852025-08-26T21:56:41.691ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
113862025-08-26T21:56:41.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
113872025-08-26T21:56:41.703ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
113882025-08-26T21:56:41.703ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.027964ms iteration = 65 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
113892025-08-26T21:56:41.730ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 44.499033ms iteration = 65 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
113902025-08-26T21:56:45.345ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:42679
113912025-08-26T21:56:45.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42679 req_id = 99ee6322-6e19-4e7e-8dfb-dcfe852265ed roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
113922025-08-26T21:56:45.345ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42679 req_id = 99ee6322-6e19-4e7e-8dfb-dcfe852265ed resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
113932025-08-26T21:56:45.348ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2828 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42679 req_id = 99ee6322-6e19-4e7e-8dfb-dcfe852265ed response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
113942025-08-26T21:56:45.348ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42679 req_id = 784489f9-4a52-44e6-a8b2-d84dea096ed8 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
113952025-08-26T21:56:45.348ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42679 req_id = 784489f9-4a52-44e6-a8b2-d84dea096ed8 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
113962025-08-26T21:56:45.350ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2390 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42679 req_id = 784489f9-4a52-44e6-a8b2-d84dea096ed8 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
113972025-08-26T21:56:45.351ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 302 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 5379064b-cf97-483a-addc-4f9869c903c6 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
113982025-08-26T21:56:46.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 130 reason = Timeout
113992025-08-26T21:56:46.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
114002025-08-26T21:56:46.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
114012025-08-26T21:56:46.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114022025-08-26T21:56:46.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
114032025-08-26T21:56:46.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
114042025-08-26T21:56:46.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
114052025-08-26T21:56:46.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
114062025-08-26T21:56:46.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.54377ms iteration = 130 status = {"chicken_switches_updated":false}
114072025-08-26T21:56:51.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 131 reason = Timeout
114082025-08-26T21:56:51.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
114092025-08-26T21:56:51.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
114102025-08-26T21:56:51.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114112025-08-26T21:56:51.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
114122025-08-26T21:56:51.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
114132025-08-26T21:56:51.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
114142025-08-26T21:56:51.673ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 66 reason = Timeout
114152025-08-26T21:56:51.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
114162025-08-26T21:56:51.673ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
114172025-08-26T21:56:51.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.278634ms iteration = 131 status = {"chicken_switches_updated":false}
114182025-08-26T21:56:51.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
114192025-08-26T21:56:51.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114202025-08-26T21:56:51.678ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
114212025-08-26T21:56:51.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
114222025-08-26T21:56:51.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
114232025-08-26T21:56:51.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
114242025-08-26T21:56:51.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
114252025-08-26T21:56:51.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114262025-08-26T21:56:51.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
114272025-08-26T21:56:51.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
114282025-08-26T21:56:51.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
114292025-08-26T21:56:51.685ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 66 reason = Timeout
114302025-08-26T21:56:51.685ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
114312025-08-26T21:56:51.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
114322025-08-26T21:56:51.703ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
114332025-08-26T21:56:51.704ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 30.073038ms iteration = 66 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
114342025-08-26T21:56:51.725ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
114352025-08-26T21:56:51.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
114362025-08-26T21:56:51.728ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 42.175871ms iteration = 66 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
114372025-08-26T21:56:55.351ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 363 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = f3cdb041-9d45-43a7-b97c-1fabdef5ee79 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
114382025-08-26T21:56:56.665ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 132 reason = Timeout
114392025-08-26T21:56:56.665ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
114402025-08-26T21:56:56.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
114412025-08-26T21:56:56.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114422025-08-26T21:56:56.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
114432025-08-26T21:56:56.673ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
114442025-08-26T21:56:56.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
114452025-08-26T21:56:56.676ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
114462025-08-26T21:56:56.676ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 11.083099ms iteration = 132 status = {"chicken_switches_updated":false}
114472025-08-26T21:57:00.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42679 req_id = 4485daea-e902-45f3-a6c8-dc9863bb049b roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
114482025-08-26T21:57:00.345ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42679 req_id = 4485daea-e902-45f3-a6c8-dc9863bb049b resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
114492025-08-26T21:57:00.349ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3868 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42679 req_id = 4485daea-e902-45f3-a6c8-dc9863bb049b response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
114502025-08-26T21:57:00.349ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42679 req_id = b69bf31f-7bb4-4de0-aba7-3b1d23911b64 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
114512025-08-26T21:57:00.349ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42679 req_id = b69bf31f-7bb4-4de0-aba7-3b1d23911b64 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
114522025-08-26T21:57:00.352ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2797 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42679 req_id = b69bf31f-7bb4-4de0-aba7-3b1d23911b64 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
114532025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
114542025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
114552025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 23 reason = Timeout
114562025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
114572025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 12 reason = Timeout
114582025-08-26T21:57:01.664ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:47:01.664344613 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
114592025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
114602025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
114612025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
114622025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 23 reason = Timeout
114632025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
114642025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
114652025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
114662025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
114672025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
114682025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 23 reason = Timeout
114692025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
114702025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
114712025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
114722025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
114732025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
114742025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
114752025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 13 reason = Timeout
114762025-08-26T21:57:01.664ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
114772025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.55\u{b5}s iteration = 13 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
114782025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
114792025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
114802025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
114812025-08-26T21:57:01.664ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1525, tv_nsec: 523830282 } })
114822025-08-26T21:57:01.664ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1525, tv_nsec: 523853422 } })
114832025-08-26T21:57:01.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
114842025-08-26T21:57:01.665ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
114852025-08-26T21:57:01.665ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.046214ms iteration = 12 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
114862025-08-26T21:57:01.665ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
114872025-08-26T21:57:01.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
114882025-08-26T21:57:01.665ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.205597ms iteration = 12 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
114892025-08-26T21:57:01.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
114902025-08-26T21:57:01.665ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 133 reason = Timeout
114912025-08-26T21:57:01.665ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
114922025-08-26T21:57:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
114932025-08-26T21:57:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
114942025-08-26T21:57:01.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
114952025-08-26T21:57:01.666ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114962025-08-26T21:57:01.667ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:47:01.664344613 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
114972025-08-26T21:57:01.667ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 2.826713ms iteration = 12 status = {"expiration":"2025-08-26T21:47:01.664344613Z","pruned":[]}
114982025-08-26T21:57:01.667ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1525, tv_nsec: 526287427 } })
114992025-08-26T21:57:01.667ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.156051ms iteration = 23 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
115002025-08-26T21:57:01.667ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115012025-08-26T21:57:01.667ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1525, tv_nsec: 526581324 } })
115022025-08-26T21:57:01.667ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
115032025-08-26T21:57:01.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115042025-08-26T21:57:01.667ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
115052025-08-26T21:57:01.668ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115062025-08-26T21:57:01.668ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
115072025-08-26T21:57:01.668ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
115082025-08-26T21:57:01.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
115092025-08-26T21:57:01.670ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
115102025-08-26T21:57:01.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
115112025-08-26T21:57:01.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
115122025-08-26T21:57:01.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
115132025-08-26T21:57:01.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
115142025-08-26T21:57:01.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
115152025-08-26T21:57:01.672ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115162025-08-26T21:57:01.672ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 15 reason = Timeout
115172025-08-26T21:57:01.672ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
115182025-08-26T21:57:01.672ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 7.38\u{b5}s iteration = 15 status = "Disabled"
115192025-08-26T21:57:01.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
115202025-08-26T21:57:01.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.920159ms iteration = 133 status = {"chicken_switches_updated":false}
115212025-08-26T21:57:01.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
115222025-08-26T21:57:01.674ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 12 reason = Timeout
115232025-08-26T21:57:01.674ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
115242025-08-26T21:57:01.674ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 25 reason = Timeout
115252025-08-26T21:57:01.674ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
115262025-08-26T21:57:01.674ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
115272025-08-26T21:57:01.674ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 23 reason = Timeout
115282025-08-26T21:57:01.674ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
115292025-08-26T21:57:01.674ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 23 reason = Timeout
115302025-08-26T21:57:01.674ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
115312025-08-26T21:57:01.674ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 23 reason = Timeout
115322025-08-26T21:57:01.674ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 23 reason = Timeout
115332025-08-26T21:57:01.674ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
115342025-08-26T21:57:01.674ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
115352025-08-26T21:57:01.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 67 reason = Timeout
115362025-08-26T21:57:01.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
115372025-08-26T21:57:01.674ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 23 reason = Timeout
115382025-08-26T21:57:01.674ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
115392025-08-26T21:57:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
115402025-08-26T21:57:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
115412025-08-26T21:57:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
115422025-08-26T21:57:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
115432025-08-26T21:57:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
115442025-08-26T21:57:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
115452025-08-26T21:57:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
115462025-08-26T21:57:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
115472025-08-26T21:57:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
115482025-08-26T21:57:01.676ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
115492025-08-26T21:57:01.676ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115502025-08-26T21:57:01.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
115512025-08-26T21:57:01.676ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115522025-08-26T21:57:01.676ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115532025-08-26T21:57:01.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115542025-08-26T21:57:01.677ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115552025-08-26T21:57:01.677ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115562025-08-26T21:57:01.680ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
115572025-08-26T21:57:01.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
115582025-08-26T21:57:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
115592025-08-26T21:57:01.682ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
115602025-08-26T21:57:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
115612025-08-26T21:57:01.682ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
115622025-08-26T21:57:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
115632025-08-26T21:57:01.682ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
115642025-08-26T21:57:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
115652025-08-26T21:57:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
115662025-08-26T21:57:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
115672025-08-26T21:57:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
115682025-08-26T21:57:01.682ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
115692025-08-26T21:57:01.682ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 8.088892ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
115702025-08-26T21:57:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
115712025-08-26T21:57:01.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
115722025-08-26T21:57:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
115732025-08-26T21:57:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
115742025-08-26T21:57:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
115752025-08-26T21:57:01.683ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
115762025-08-26T21:57:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
115772025-08-26T21:57:01.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
115782025-08-26T21:57:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
115792025-08-26T21:57:01.685ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
115802025-08-26T21:57:01.685ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
115812025-08-26T21:57:01.685ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 12 reason = Timeout
115822025-08-26T21:57:01.685ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 23 reason = Timeout
115832025-08-26T21:57:01.685ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
115842025-08-26T21:57:01.685ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 67 reason = Timeout
115852025-08-26T21:57:01.685ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
115862025-08-26T21:57:01.685ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
115872025-08-26T21:57:01.685ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
115882025-08-26T21:57:01.685ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 23 reason = Timeout
115892025-08-26T21:57:01.685ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
115902025-08-26T21:57:01.685ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1525, tv_nsec: 544665361 } })
115912025-08-26T21:57:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
115922025-08-26T21:57:01.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
115932025-08-26T21:57:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
115942025-08-26T21:57:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
115952025-08-26T21:57:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
115962025-08-26T21:57:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
115972025-08-26T21:57:01.686ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
115982025-08-26T21:57:01.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115992025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 12 reason = Timeout
116002025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
116012025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 23 reason = Timeout
116022025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
116032025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 23 reason = Timeout
116042025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
116052025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 12 reason = Timeout
116062025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 12 reason = Timeout
116072025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
116082025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
116092025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 23 reason = Timeout
116102025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 10.9\u{b5}s iteration = 23 status = {"disabled":true,"errors":[],"sps":[]}
116112025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 23 reason = Timeout
116122025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 23 reason = Timeout
116132025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
116142025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 23 reason = Timeout
116152025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
116162025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
116172025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 23 reason = Timeout
116182025-08-26T21:57:01.687ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
116192025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
116202025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 12 reason = Timeout
116212025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
116222025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
116232025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
116242025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
116252025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
116262025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
116272025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
116282025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
116292025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
116302025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
116312025-08-26T21:57:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
116322025-08-26T21:57:01.688ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116332025-08-26T21:57:01.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 3.126091ms iteration = 23 status = {"errors":[],"garbage_collect_requested":[]}
116342025-08-26T21:57:01.690ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
116352025-08-26T21:57:01.690ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116362025-08-26T21:57:01.690ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 3.357456ms iteration = 12 status = {"by_rx":{},"error":null}
116372025-08-26T21:57:01.691ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
116382025-08-26T21:57:01.691ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
116392025-08-26T21:57:01.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
116402025-08-26T21:57:01.692ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
116412025-08-26T21:57:01.692ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
116422025-08-26T21:57:01.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
116432025-08-26T21:57:01.693ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 6.417095ms iteration = 12 status = {"errors":[],"found_port_ok":[]}
116442025-08-26T21:57:01.695ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
116452025-08-26T21:57:01.695ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
116462025-08-26T21:57:01.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
116472025-08-26T21:57:01.703ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
116482025-08-26T21:57:01.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
116492025-08-26T21:57:01.751ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
116502025-08-26T21:57:01.751ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 77.125348ms iteration = 67 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
116512025-08-26T21:57:01.755ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
116522025-08-26T21:57:01.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
116532025-08-26T21:57:01.756ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116542025-08-26T21:57:01.757ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116552025-08-26T21:57:01.757ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 70.653233ms iteration = 12 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
116562025-08-26T21:57:01.758ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
116572025-08-26T21:57:01.758ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
116582025-08-26T21:57:01.758ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
116592025-08-26T21:57:01.758ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
116602025-08-26T21:57:01.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
116612025-08-26T21:57:01.760ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
116622025-08-26T21:57:01.760ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
116632025-08-26T21:57:01.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
116642025-08-26T21:57:01.760ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116652025-08-26T21:57:01.761ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
116662025-08-26T21:57:01.761ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
116672025-08-26T21:57:01.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
116682025-08-26T21:57:01.766ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
116692025-08-26T21:57:01.766ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
116702025-08-26T21:57:01.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
116712025-08-26T21:57:01.770ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
116722025-08-26T21:57:01.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
116732025-08-26T21:57:01.772ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
116742025-08-26T21:57:01.772ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
116752025-08-26T21:57:01.772ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
116762025-08-26T21:57:01.772ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116772025-08-26T21:57:01.772ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
116782025-08-26T21:57:01.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
116792025-08-26T21:57:01.773ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
116802025-08-26T21:57:01.773ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
116812025-08-26T21:57:01.773ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
116822025-08-26T21:57:01.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
116832025-08-26T21:57:01.773ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116842025-08-26T21:57:01.773ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116852025-08-26T21:57:01.775ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 88.166798ms iteration = 23 status = {"errors":[],"finish_invoked_ok":[]}
116862025-08-26T21:57:01.775ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
116872025-08-26T21:57:01.775ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
116882025-08-26T21:57:01.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
116892025-08-26T21:57:01.778ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
116902025-08-26T21:57:01.778ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 103.406171ms iteration = 23 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
116912025-08-26T21:57:01.778ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
116922025-08-26T21:57:01.778ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
116932025-08-26T21:57:01.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
116942025-08-26T21:57:01.779ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
116952025-08-26T21:57:01.779ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
116962025-08-26T21:57:01.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
116972025-08-26T21:57:01.780ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
116982025-08-26T21:57:01.780ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
116992025-08-26T21:57:01.780ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 105.700712ms iteration = 12 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
117002025-08-26T21:57:01.781ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
117012025-08-26T21:57:01.781ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
117022025-08-26T21:57:01.781ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 94.563332ms iteration = 12 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
117032025-08-26T21:57:01.782ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
117042025-08-26T21:57:01.782ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
117052025-08-26T21:57:01.782ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
117062025-08-26T21:57:01.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
117072025-08-26T21:57:01.784ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
117082025-08-26T21:57:01.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
117092025-08-26T21:57:01.786ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
117102025-08-26T21:57:01.786ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
117112025-08-26T21:57:01.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
117122025-08-26T21:57:01.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
117132025-08-26T21:57:01.789ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117142025-08-26T21:57:01.790ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 103.239997ms iteration = 23 status = {"errors":[],"requests_created_ok":[]}
117152025-08-26T21:57:01.791ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
117162025-08-26T21:57:01.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
117172025-08-26T21:57:01.791ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 117.362096ms iteration = 25 status = {"physical_disks_added":0}
117182025-08-26T21:57:01.793ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
117192025-08-26T21:57:01.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
117202025-08-26T21:57:01.793ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
117212025-08-26T21:57:01.793ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
117222025-08-26T21:57:01.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
117232025-08-26T21:57:01.796ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 122.275417ms iteration = 23 status = 0
117242025-08-26T21:57:01.847ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117252025-08-26T21:57:01.851ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-08-26T21:45:59.866217Z, time_modified: 2025-08-26T21:45:59.866217Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
117262025-08-26T21:57:01.851ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 187.531007ms iteration = 12 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]}},"default_endpoint":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
117272025-08-26T21:57:01.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
117282025-08-26T21:57:01.853ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
117292025-08-26T21:57:01.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
117302025-08-26T21:57:01.856ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
117312025-08-26T21:57:01.856ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
117322025-08-26T21:57:01.856ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-08-26 21:46:04.065217 UTC dns_group = internal generation = 1
117332025-08-26T21:57:01.856ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 191.923397ms iteration = 12 status = {"generation":1}
117342025-08-26T21:57:01.856ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
117352025-08-26T21:57:01.856ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
117362025-08-26T21:57:01.856ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
117372025-08-26T21:57:01.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
117382025-08-26T21:57:01.858ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
117392025-08-26T21:57:01.859ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
117402025-08-26T21:57:01.859ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 0 version = 2
117412025-08-26T21:57:01.859ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
117422025-08-26T21:57:01.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
117432025-08-26T21:57:01.860ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
117442025-08-26T21:57:01.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
117452025-08-26T21:57:01.860ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 175.297822ms iteration = 67 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
117462025-08-26T21:57:01.861ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117472025-08-26T21:57:01.865ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 177.698576ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
117482025-08-26T21:57:01.865ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
117492025-08-26T21:57:01.865ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
117502025-08-26T21:57:01.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
117512025-08-26T21:57:01.867ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 4 version = 2
117522025-08-26T21:57:01.867ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
117532025-08-26T21:57:01.867ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-08-26 21:46:05.296205 UTC dns_group = external generation = 2
117542025-08-26T21:57:01.867ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 203.16125ms iteration = 12 status = {"generation":2}
117552025-08-26T21:57:01.891ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
117562025-08-26T21:57:01.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
117572025-08-26T21:57:01.907ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
117582025-08-26T21:57:01.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
117592025-08-26T21:57:01.914ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
117602025-08-26T21:57:01.914ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
117612025-08-26T21:57:01.914ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 229.203217ms iteration = 23 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
117622025-08-26T21:57:01.929ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 242.011276ms iteration = 23 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
117632025-08-26T21:57:01.941ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
117642025-08-26T21:57:01.941ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
117652025-08-26T21:57:01.942ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d292dcd6-8e03-45ba-8b43-e0c7f7ee498e", "content-length": "26", "date": "Tue, 26 Aug 2025 21:57:01 GMT"} })
117662025-08-26T21:57:01.942ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
117672025-08-26T21:57:01.944ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
117682025-08-26T21:57:01.946ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
117692025-08-26T21:57:01.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
117702025-08-26T21:57:01.948ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117712025-08-26T21:57:01.950ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
117722025-08-26T21:57:01.950ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
117732025-08-26T21:57:01.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
117742025-08-26T21:57:01.977ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = instance_watcher body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vmms/b931750f-371f-4349-9ab8-e4868c2650e3/state
117752025-08-26T21:57:01.981ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = instance_watcher result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vmms/b931750f-371f-4349-9ab8-e4868c2650e3/state", status: 200, headers: {"content-type": "application/json", "x-request-id": "232628bf-755f-414e-a57e-bb0922129b4a", "content-length": "130", "date": "Tue, 26 Aug 2025 21:57:01 GMT"} })
117762025-08-26T21:57:01.983ZDEBGnexus (BackgroundTasks): updating instance state background_task = instance_watcher instance_id = 5f8711b0-0cf9-4ef2-aaaa-3cc9f32ebe3c sled_id = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa state = SledVmmState { vmm_state: VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2025-08-26T21:57:01.346616291Z }, migration_in: None, migration_out: None } vmm_id = b931750f-371f-4349-9ab8-e4868c2650e3
117772025-08-26T21:57:01.983ZINFOnexus (BackgroundTasks): received new VMM runtime state from sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:2140 instance_id = 5f8711b0-0cf9-4ef2-aaaa-3cc9f32ebe3c migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = b931750f-371f-4349-9ab8-e4868c2650e3 sled_id = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa vmm_id = b931750f-371f-4349-9ab8-e4868c2650e3 vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2025-08-26T21:57:01.346616291Z }
117782025-08-26T21:57:01.983ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = 5f8711b0-0cf9-4ef2-aaaa-3cc9f32ebe3c roles = RoleSet { roles: {} } sled_id = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa vmm_id = b931750f-371f-4349-9ab8-e4868c2650e3
117792025-08-26T21:57:01.984ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher instance_id = 5f8711b0-0cf9-4ef2-aaaa-3cc9f32ebe3c resource = Database result = Ok(()) sled_id = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa vmm_id = b931750f-371f-4349-9ab8-e4868c2650e3
117802025-08-26T21:57:01.984ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
117812025-08-26T21:57:01.984ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
117822025-08-26T21:57:01.984ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3fc97791-f0de-462f-a0f7-89211de566f0", "content-length": "26", "date": "Tue, 26 Aug 2025 21:57:01 GMT"} })
117832025-08-26T21:57:01.984ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
117842025-08-26T21:57:01.986ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
117852025-08-26T21:57:01.990ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
117862025-08-26T21:57:01.992ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
117872025-08-26T21:57:01.992ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
117882025-08-26T21:57:01.993ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6426abd-90ce-404a-8f60-b5683adff95f", "content-length": "26", "date": "Tue, 26 Aug 2025 21:57:01 GMT"} })
117892025-08-26T21:57:01.993ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
117902025-08-26T21:57:01.994ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
117912025-08-26T21:57:01.999ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117922025-08-26T21:57:02.002ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
117932025-08-26T21:57:02.002ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
117942025-08-26T21:57:02.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
117952025-08-26T21:57:02.002ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1 update_sagas_queued = 0
117962025-08-26T21:57:02.004ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 318.434138ms iteration = 23 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{"running":1},"pruned_instances":0,"total_instances":1,"update_sagas_queued":0}
117972025-08-26T21:57:02.032ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
117982025-08-26T21:57:02.032ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 346.381258ms iteration = 12 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
117992025-08-26T21:57:02.180ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
118002025-08-26T21:57:02.199ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
118012025-08-26T21:57:02.200ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "ec0f11b2-e98e-41d1-b927-b6160743c0c5", "date": "Tue, 26 Aug 2025 21:57:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
118022025-08-26T21:57:02.200ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
118032025-08-26T21:57:02.202ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "354c9f23-ec5f-4d0a-9a5c-2d28baf66754", "date": "Tue, 26 Aug 2025 21:57:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
118042025-08-26T21:57:02.202ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
118052025-08-26T21:57:02.204ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
118062025-08-26T21:57:02.204ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 540.354581ms iteration = 14 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
118072025-08-26T21:57:02.232ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
118082025-08-26T21:57:02.232ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
118092025-08-26T21:57:02.233ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "711576c6-76fa-4ebb-9868-0798088a06b8", "content-length": "583", "date": "Tue, 26 Aug 2025 21:57:01 GMT"} })
118102025-08-26T21:57:02.233ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
118112025-08-26T21:57:02.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
118122025-08-26T21:57:02.247ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
118132025-08-26T21:57:02.250ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "d92c49db-68a7-4dce-a812-854416151471", "content-length": "810", "date": "Tue, 26 Aug 2025 21:57:01 GMT"} })
118142025-08-26T21:57:02.250ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {VirtualNetworkInterfaceHost { physical_host_ip: fd00:1122:3344:101::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), vni: Vni(8860997) }} sled = sock
118152025-08-26T21:57:02.250ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
118162025-08-26T21:57:02.250ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
118172025-08-26T21:57:02.252ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 577.787055ms iteration = 23 status = {}
118182025-08-26T21:57:02.260ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
118192025-08-26T21:57:02.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
118202025-08-26T21:57:02.263ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 598.663335ms iteration = 23 status = null
118212025-08-26T21:57:02.292ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {a751b6c0-c32c-4656-871a-c6c5c7b72e38: {10.151.2.176: {31e4670d-43eb-47bb-9561-246959f0a73c}, 10.151.2.175: {31e4670d-43eb-47bb-9561-246959f0a73c}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
118222025-08-26T21:57:02.292ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
118232025-08-26T21:57:02.293ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "dc601a11-a7d4-4276-b996-55ed5c332737", "date": "Tue, 26 Aug 2025 21:57:01 GMT"} })
118242025-08-26T21:57:02.293ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
118252025-08-26T21:57:02.293ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 628.996388ms iteration = 23 status = {}
118262025-08-26T21:57:02.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
118272025-08-26T21:57:02.295ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
118282025-08-26T21:57:02.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
118292025-08-26T21:57:02.298ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
118302025-08-26T21:57:02.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
118312025-08-26T21:57:02.301ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
118322025-08-26T21:57:02.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
118332025-08-26T21:57:02.304ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
118342025-08-26T21:57:02.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
118352025-08-26T21:57:02.308ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
118362025-08-26T21:57:02.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
118372025-08-26T21:57:02.312ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
118382025-08-26T21:57:02.312ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
118392025-08-26T21:57:02.312ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
118402025-08-26T21:57:02.317ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "cdbdacef-1df3-49a4-a1e8-251fcc0a97b1", "date": "Tue, 26 Aug 2025 21:57:02 GMT"} })
118412025-08-26T21:57:02.320ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 632.655001ms iteration = 23 status = {}
118422025-08-26T21:57:02.328ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
118432025-08-26T21:57:02.329ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "bf0821df-1f41-4df4-8b3d-d699dc1d06b8", "date": "Tue, 26 Aug 2025 21:57:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
118442025-08-26T21:57:02.329ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
118452025-08-26T21:57:02.331ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "962a5cf7-2fd6-4962-ac08-a71d6043ae3d", "date": "Tue, 26 Aug 2025 21:57:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
118462025-08-26T21:57:02.331ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
118472025-08-26T21:57:02.333ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "883ced30-8bf5-408c-a908-387f44668db9", "date": "Tue, 26 Aug 2025 21:57:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
118482025-08-26T21:57:02.333ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
118492025-08-26T21:57:02.335ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
118502025-08-26T21:57:02.335ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 671.504657ms iteration = 14 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
118512025-08-26T21:57:02.343ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
118522025-08-26T21:57:02.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
118532025-08-26T21:57:02.345ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
118542025-08-26T21:57:02.345ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
118552025-08-26T21:57:02.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
118562025-08-26T21:57:02.358ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
118572025-08-26T21:57:02.358ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
118582025-08-26T21:57:02.358ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
118592025-08-26T21:57:02.358ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
118602025-08-26T21:57:02.358ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
118612025-08-26T21:57:02.358ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
118622025-08-26T21:57:02.358ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
118632025-08-26T21:57:02.358ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
118642025-08-26T21:57:02.359ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
118652025-08-26T21:57:02.359ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
118662025-08-26T21:57:02.359ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
118672025-08-26T21:57:02.359ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
118682025-08-26T21:57:02.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
118692025-08-26T21:57:02.362ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
118702025-08-26T21:57:02.362ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
118712025-08-26T21:57:02.483ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 4
118722025-08-26T21:57:02.483ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221
118732025-08-26T21:57:02.515ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
118742025-08-26T21:57:02.515ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::a]:36588
118752025-08-26T21:57:02.515ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "272acb58-e440-4c33-8957-bc85bbc2637b", "date": "Tue, 26 Aug 2025 21:57:02 GMT"} })
118762025-08-26T21:57:02.515ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
118772025-08-26T21:57:02.515ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
118782025-08-26T21:57:02.515ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:36588 req_id = 6a1c72b6-8365-45eb-9a1b-d23e17b36ab6 roles = RoleSet { roles: {} } uri = /metrics/producers
118792025-08-26T21:57:02.516ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:36588 req_id = 6a1c72b6-8365-45eb-9a1b-d23e17b36ab6 resource = Database result = Ok(()) uri = /metrics/producers
118802025-08-26T21:57:02.516ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "5456f36b-f8c4-4696-be1e-4fd5fee5d087", "content-length": "682", "date": "Tue, 26 Aug 2025 21:57:02 GMT"} })
118812025-08-26T21:57:02.516ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
118822025-08-26T21:57:02.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
118832025-08-26T21:57:02.520ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
118842025-08-26T21:57:02.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
118852025-08-26T21:57:02.524ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
118862025-08-26T21:57:02.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
118872025-08-26T21:57:02.526ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
118882025-08-26T21:57:02.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
118892025-08-26T21:57:02.534ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 859.393633ms iteration = 23 status = {}
118902025-08-26T21:57:02.561ZINFOnexus: assigned collector to new producer collector_id = f843ff0e-eef8-49c4-b428-23e15df1def3 file = nexus/src/app/oximeter.rs:76 producer_id = 4996dcc4-c530-4f64-b407-fa372462bd9e
118912025-08-26T21:57:02.561ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 45259 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:36588 req_id = 6a1c72b6-8365-45eb-9a1b-d23e17b36ab6 response_code = 201 uri = /metrics/producers
118922025-08-26T21:57:02.561ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "6a1c72b6-8365-45eb-9a1b-d23e17b36ab6", "content-length": "41", "date": "Tue, 26 Aug 2025 21:57:02 GMT"} })
118932025-08-26T21:57:02.562ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
118942025-08-26T21:57:05.355ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3251 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 5236a31c-3c85-4bb8-9809-2bbbc438d96c response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
118952025-08-26T21:57:06.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 134 reason = Timeout
118962025-08-26T21:57:06.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
118972025-08-26T21:57:06.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
118982025-08-26T21:57:06.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118992025-08-26T21:57:06.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
119002025-08-26T21:57:06.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
119012025-08-26T21:57:06.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
119022025-08-26T21:57:06.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
119032025-08-26T21:57:06.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.96459ms iteration = 134 status = {"chicken_switches_updated":false}
119042025-08-26T21:57:11.665ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 135 reason = Timeout
119052025-08-26T21:57:11.665ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
119062025-08-26T21:57:11.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
119072025-08-26T21:57:11.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119082025-08-26T21:57:11.670ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
119092025-08-26T21:57:11.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
119102025-08-26T21:57:11.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
119112025-08-26T21:57:11.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
119122025-08-26T21:57:11.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.583221ms iteration = 135 status = {"chicken_switches_updated":false}
119132025-08-26T21:57:11.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 68 reason = Timeout
119142025-08-26T21:57:11.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
119152025-08-26T21:57:11.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
119162025-08-26T21:57:11.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119172025-08-26T21:57:11.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
119182025-08-26T21:57:11.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
119192025-08-26T21:57:11.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
119202025-08-26T21:57:11.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
119212025-08-26T21:57:11.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
119222025-08-26T21:57:11.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119232025-08-26T21:57:11.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
119242025-08-26T21:57:11.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
119252025-08-26T21:57:11.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
119262025-08-26T21:57:11.685ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 68 reason = Timeout
119272025-08-26T21:57:11.685ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
119282025-08-26T21:57:11.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
119292025-08-26T21:57:11.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
119302025-08-26T21:57:11.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
119312025-08-26T21:57:11.699ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
119322025-08-26T21:57:11.699ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.82437ms iteration = 68 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
119332025-08-26T21:57:11.725ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 40.052433ms iteration = 68 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
119342025-08-26T21:57:15.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 405 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = ceef65cb-8571-4ea0-959d-58962d648b57 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
119352025-08-26T21:57:16.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 136 reason = Timeout
119362025-08-26T21:57:16.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
119372025-08-26T21:57:16.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
119382025-08-26T21:57:16.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119392025-08-26T21:57:16.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
119402025-08-26T21:57:16.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
119412025-08-26T21:57:16.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
119422025-08-26T21:57:16.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
119432025-08-26T21:57:16.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.674613ms iteration = 136 status = {"chicken_switches_updated":false}
119442025-08-26T21:57:21.665ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 137 reason = Timeout
119452025-08-26T21:57:21.665ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
119462025-08-26T21:57:21.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
119472025-08-26T21:57:21.669ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119482025-08-26T21:57:21.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
119492025-08-26T21:57:21.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
119502025-08-26T21:57:21.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
119512025-08-26T21:57:21.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
119522025-08-26T21:57:21.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.275926ms iteration = 137 status = {"chicken_switches_updated":false}
119532025-08-26T21:57:21.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 69 reason = Timeout
119542025-08-26T21:57:21.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
119552025-08-26T21:57:21.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
119562025-08-26T21:57:21.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119572025-08-26T21:57:21.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
119582025-08-26T21:57:21.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
119592025-08-26T21:57:21.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
119602025-08-26T21:57:21.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
119612025-08-26T21:57:21.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
119622025-08-26T21:57:21.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119632025-08-26T21:57:21.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
119642025-08-26T21:57:21.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
119652025-08-26T21:57:21.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
119662025-08-26T21:57:21.685ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 69 reason = Timeout
119672025-08-26T21:57:21.685ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
119682025-08-26T21:57:21.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
119692025-08-26T21:57:21.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
119702025-08-26T21:57:21.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
119712025-08-26T21:57:21.689ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.718284ms iteration = 69 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
119722025-08-26T21:57:21.699ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
119732025-08-26T21:57:21.699ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.073855ms iteration = 69 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
119742025-08-26T21:57:25.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 450 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 349e680b-0e6f-47be-892c-e5dc976793a7 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
119752025-08-26T21:57:26.665ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 138 reason = Timeout
119762025-08-26T21:57:26.665ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
119772025-08-26T21:57:26.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
119782025-08-26T21:57:26.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119792025-08-26T21:57:26.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
119802025-08-26T21:57:26.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
119812025-08-26T21:57:26.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
119822025-08-26T21:57:26.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
119832025-08-26T21:57:26.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.792206ms iteration = 138 status = {"chicken_switches_updated":false}
119842025-08-26T21:57:30.345ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42679 req_id = 1850a634-ef83-40f2-894b-f9d8fbf9a3f9 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
119852025-08-26T21:57:30.345ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42679 req_id = 1850a634-ef83-40f2-894b-f9d8fbf9a3f9 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
119862025-08-26T21:57:30.348ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3690 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42679 req_id = 1850a634-ef83-40f2-894b-f9d8fbf9a3f9 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?sort_by=id_ascending
119872025-08-26T21:57:30.349ZDEBGnexus (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42679 req_id = faa3328b-ba31-452d-9919-cb904151e7f0 roles = RoleSet { roles: {} } uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
119882025-08-26T21:57:30.349ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42679 req_id = faa3328b-ba31-452d-9919-cb904151e7f0 resource = Database result = Ok(()) uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
119892025-08-26T21:57:30.351ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2660 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42679 req_id = faa3328b-ba31-452d-9919-cb904151e7f0 response_code = 200 uri = /metrics/collectors/f843ff0e-eef8-49c4-b428-23e15df1def3/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMzhjYTg4YS1lZGE1LTQwZGEtOGE4MS1hN2UzMzdhMmM4ZmIifX0%3D
119902025-08-26T21:57:31.664ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 24 reason = Timeout
119912025-08-26T21:57:31.664ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 24 reason = Timeout
119922025-08-26T21:57:31.664ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 24 reason = Timeout
119932025-08-26T21:57:31.664ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
119942025-08-26T21:57:31.664ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
119952025-08-26T21:57:31.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
119962025-08-26T21:57:31.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
119972025-08-26T21:57:31.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 139 reason = Timeout
119982025-08-26T21:57:31.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
119992025-08-26T21:57:31.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
120002025-08-26T21:57:31.666ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1555, tv_nsec: 525980890 } })
120012025-08-26T21:57:31.667ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
120022025-08-26T21:57:31.667ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1555, tv_nsec: 526566613 } })
120032025-08-26T21:57:31.667ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.566361ms iteration = 24 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
120042025-08-26T21:57:31.667ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
120052025-08-26T21:57:31.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120062025-08-26T21:57:31.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
120072025-08-26T21:57:31.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
120082025-08-26T21:57:31.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
120092025-08-26T21:57:31.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
120102025-08-26T21:57:31.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.890085ms iteration = 139 status = {"chicken_switches_updated":false}
120112025-08-26T21:57:31.674ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 24 reason = Timeout
120122025-08-26T21:57:31.674ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
120132025-08-26T21:57:31.674ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 24 reason = Timeout
120142025-08-26T21:57:31.674ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 24 reason = Timeout
120152025-08-26T21:57:31.674ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
120162025-08-26T21:57:31.674ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
120172025-08-26T21:57:31.674ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 26 reason = Timeout
120182025-08-26T21:57:31.674ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
120192025-08-26T21:57:31.674ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 24 reason = Timeout
120202025-08-26T21:57:31.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 70 reason = Timeout
120212025-08-26T21:57:31.674ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
120222025-08-26T21:57:31.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
120232025-08-26T21:57:31.674ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 24 reason = Timeout
120242025-08-26T21:57:31.674ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
120252025-08-26T21:57:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
120262025-08-26T21:57:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
120272025-08-26T21:57:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
120282025-08-26T21:57:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
120292025-08-26T21:57:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
120302025-08-26T21:57:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
120312025-08-26T21:57:31.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
120322025-08-26T21:57:31.676ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120332025-08-26T21:57:31.676ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120342025-08-26T21:57:31.676ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120352025-08-26T21:57:31.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120362025-08-26T21:57:31.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
120372025-08-26T21:57:31.679ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
120382025-08-26T21:57:31.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
120392025-08-26T21:57:31.679ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
120402025-08-26T21:57:31.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
120412025-08-26T21:57:31.680ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
120422025-08-26T21:57:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
120432025-08-26T21:57:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
120442025-08-26T21:57:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
120452025-08-26T21:57:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
120462025-08-26T21:57:31.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
120472025-08-26T21:57:31.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
120482025-08-26T21:57:31.681ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1555, tv_nsec: 541047199 } })
120492025-08-26T21:57:31.682ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
120502025-08-26T21:57:31.682ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
120512025-08-26T21:57:31.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
120522025-08-26T21:57:31.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
120532025-08-26T21:57:31.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
120542025-08-26T21:57:31.684ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120552025-08-26T21:57:31.686ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 24 reason = Timeout
120562025-08-26T21:57:31.686ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
120572025-08-26T21:57:31.686ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 24 reason = Timeout
120582025-08-26T21:57:31.686ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
120592025-08-26T21:57:31.686ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
120602025-08-26T21:57:31.686ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 24 reason = Timeout
120612025-08-26T21:57:31.686ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
120622025-08-26T21:57:31.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 70 reason = Timeout
120632025-08-26T21:57:31.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
120642025-08-26T21:57:31.686ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 24 reason = Timeout
120652025-08-26T21:57:31.686ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
120662025-08-26T21:57:31.686ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 24 reason = Timeout
120672025-08-26T21:57:31.686ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
120682025-08-26T21:57:31.686ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
120692025-08-26T21:57:31.686ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 24 reason = Timeout
120702025-08-26T21:57:31.686ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
120712025-08-26T21:57:31.686ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 24 reason = Timeout
120722025-08-26T21:57:31.686ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
120732025-08-26T21:57:31.686ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 24 reason = Timeout
120742025-08-26T21:57:31.686ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 24 reason = Timeout
120752025-08-26T21:57:31.686ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
120762025-08-26T21:57:31.686ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 10.24\u{b5}s iteration = 24 status = {"disabled":true,"errors":[],"sps":[]}
120772025-08-26T21:57:31.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
120782025-08-26T21:57:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
120792025-08-26T21:57:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
120802025-08-26T21:57:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
120812025-08-26T21:57:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
120822025-08-26T21:57:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
120832025-08-26T21:57:31.687ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
120842025-08-26T21:57:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
120852025-08-26T21:57:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
120862025-08-26T21:57:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
120872025-08-26T21:57:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
120882025-08-26T21:57:31.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
120892025-08-26T21:57:31.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
120902025-08-26T21:57:31.689ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120912025-08-26T21:57:31.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
120922025-08-26T21:57:31.692ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
120932025-08-26T21:57:31.692ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
120942025-08-26T21:57:31.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
120952025-08-26T21:57:31.693ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 6.836904ms iteration = 24 status = {"errors":[],"requests_created_ok":[]}
120962025-08-26T21:57:31.702ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
120972025-08-26T21:57:31.703ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 16.231026ms iteration = 24 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":1,"total_instances":0,"update_sagas_queued":0}
120982025-08-26T21:57:31.743ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
120992025-08-26T21:57:31.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
121002025-08-26T21:57:31.754ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121012025-08-26T21:57:31.757ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
121022025-08-26T21:57:31.757ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
121032025-08-26T21:57:31.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121042025-08-26T21:57:31.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
121052025-08-26T21:57:31.759ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
121062025-08-26T21:57:31.759ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121072025-08-26T21:57:31.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
121082025-08-26T21:57:31.760ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
121092025-08-26T21:57:31.760ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
121102025-08-26T21:57:31.760ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
121112025-08-26T21:57:31.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
121122025-08-26T21:57:31.761ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
121132025-08-26T21:57:31.761ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
121142025-08-26T21:57:31.761ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "968c9c36-2a8b-4638-95cc-7111733ff9e9", "content-length": "26", "date": "Tue, 26 Aug 2025 21:57:31 GMT"} })
121152025-08-26T21:57:31.762ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
121162025-08-26T21:57:31.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
121172025-08-26T21:57:31.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
121182025-08-26T21:57:31.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
121192025-08-26T21:57:31.762ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
121202025-08-26T21:57:31.762ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
121212025-08-26T21:57:31.763ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121222025-08-26T21:57:31.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
121232025-08-26T21:57:31.763ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
121242025-08-26T21:57:31.763ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
121252025-08-26T21:57:31.763ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
121262025-08-26T21:57:31.763ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
121272025-08-26T21:57:31.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
121282025-08-26T21:57:31.765ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
121292025-08-26T21:57:31.766ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
121302025-08-26T21:57:31.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
121312025-08-26T21:57:31.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
121322025-08-26T21:57:31.767ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
121332025-08-26T21:57:31.767ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
121342025-08-26T21:57:31.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
121352025-08-26T21:57:31.769ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 82.451678ms iteration = 24 status = {"errors":[],"finish_invoked_ok":[]}
121362025-08-26T21:57:31.770ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
121372025-08-26T21:57:31.770ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
121382025-08-26T21:57:31.770ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 84.236139ms iteration = 24 status = {"errors":[],"garbage_collect_requested":[]}
121392025-08-26T21:57:31.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
121402025-08-26T21:57:31.771ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 84.474404ms iteration = 70 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
121412025-08-26T21:57:31.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
121422025-08-26T21:57:31.772ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 86.015299ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
121432025-08-26T21:57:31.785ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
121442025-08-26T21:57:31.785ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 111.348ms iteration = 70 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
121452025-08-26T21:57:31.814ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 139.729099ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
121462025-08-26T21:57:31.821ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
121472025-08-26T21:57:31.821ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
121482025-08-26T21:57:31.821ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 135.262609ms iteration = 24 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
121492025-08-26T21:57:31.827ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121502025-08-26T21:57:31.827ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
121512025-08-26T21:57:31.827ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
121522025-08-26T21:57:31.827ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "76860766-0268-4200-9765-cb94fb89c2b2", "content-length": "26", "date": "Tue, 26 Aug 2025 21:57:31 GMT"} })
121532025-08-26T21:57:31.827ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
121542025-08-26T21:57:31.828ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
121552025-08-26T21:57:31.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
121562025-08-26T21:57:31.829ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
121572025-08-26T21:57:31.829ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
121582025-08-26T21:57:31.829ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
121592025-08-26T21:57:31.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
121602025-08-26T21:57:31.831ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 145.004409ms iteration = 24 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
121612025-08-26T21:57:31.834ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
121622025-08-26T21:57:31.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
121632025-08-26T21:57:31.837ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
121642025-08-26T21:57:31.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
121652025-08-26T21:57:31.839ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121662025-08-26T21:57:31.840ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
121672025-08-26T21:57:31.840ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
121682025-08-26T21:57:31.841ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7a8f038-44ba-4d4d-93b4-e6688ba5f459", "content-length": "26", "date": "Tue, 26 Aug 2025 21:57:31 GMT"} })
121692025-08-26T21:57:31.841ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
121702025-08-26T21:57:31.842ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
121712025-08-26T21:57:31.842ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
121722025-08-26T21:57:31.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
121732025-08-26T21:57:31.843ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
121742025-08-26T21:57:31.844ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 169.783647ms iteration = 24 status = 0
121752025-08-26T21:57:31.849ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 175.211489ms iteration = 26 status = {"physical_disks_added":0}
121762025-08-26T21:57:31.867ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
121772025-08-26T21:57:31.867ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 192.610972ms iteration = 24 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
121782025-08-26T21:57:31.890ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
121792025-08-26T21:57:31.890ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
121802025-08-26T21:57:31.890ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a2a7569-8c05-4120-84c9-fb8037f591fd", "content-length": "372", "date": "Tue, 26 Aug 2025 21:57:31 GMT"} })
121812025-08-26T21:57:31.890ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
121822025-08-26T21:57:31.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
121832025-08-26T21:57:31.902ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
121842025-08-26T21:57:31.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
121852025-08-26T21:57:31.904ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121862025-08-26T21:57:31.904ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
121872025-08-26T21:57:31.904ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
121882025-08-26T21:57:31.905ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "722b0dff-8a72-4205-adeb-5fdff03dc28e", "date": "Tue, 26 Aug 2025 21:57:31 GMT"} })
121892025-08-26T21:57:31.905ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
121902025-08-26T21:57:31.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
121912025-08-26T21:57:31.906ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
121922025-08-26T21:57:31.906ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
121932025-08-26T21:57:31.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
121942025-08-26T21:57:31.907ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121952025-08-26T21:57:31.907ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
121962025-08-26T21:57:31.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
121972025-08-26T21:57:31.909ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
121982025-08-26T21:57:31.909ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
121992025-08-26T21:57:31.909ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 245.613257ms iteration = 24 status = {}
122002025-08-26T21:57:31.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
122012025-08-26T21:57:31.910ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
122022025-08-26T21:57:31.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
122032025-08-26T21:57:31.914ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
122042025-08-26T21:57:31.914ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
122052025-08-26T21:57:31.915ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "e7f79188-69fb-4a80-b1c2-cf4c90244451", "date": "Tue, 26 Aug 2025 21:57:31 GMT"} })
122062025-08-26T21:57:31.916ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 230.076277ms iteration = 24 status = {}
122072025-08-26T21:57:31.943ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
122082025-08-26T21:57:31.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
122092025-08-26T21:57:31.945ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 281.609538ms iteration = 24 status = null
122102025-08-26T21:57:31.989ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
122112025-08-26T21:57:31.989ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "335204ff-bd69-4bf8-a9a3-4153c5c7d429", "content-length": "691", "date": "Tue, 26 Aug 2025 21:57:31 GMT"} })
122122025-08-26T21:57:31.989ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
122132025-08-26T21:57:31.989ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
122142025-08-26T21:57:31.989ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
122152025-08-26T21:57:31.991ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 317.207311ms iteration = 24 status = {}
122162025-08-26T21:57:32.024ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
122172025-08-26T21:57:32.024ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
122182025-08-26T21:57:32.025ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
122192025-08-26T21:57:32.026ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
122202025-08-26T21:57:32.026ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
122212025-08-26T21:57:32.038ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
122222025-08-26T21:57:32.038ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
122232025-08-26T21:57:32.038ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
122242025-08-26T21:57:32.038ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
122252025-08-26T21:57:32.038ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
122262025-08-26T21:57:32.038ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
122272025-08-26T21:57:32.038ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
122282025-08-26T21:57:32.038ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
122292025-08-26T21:57:32.039ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
122302025-08-26T21:57:32.039ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
122312025-08-26T21:57:32.039ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
122322025-08-26T21:57:32.039ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
122332025-08-26T21:57:32.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
122342025-08-26T21:57:32.041ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
122352025-08-26T21:57:32.042ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
122362025-08-26T21:57:32.227ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "c08a4d83-d4c2-4323-a7c8-a82670fb0535", "date": "Tue, 26 Aug 2025 21:57:31 GMT"} })
122372025-08-26T21:57:32.227ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
122382025-08-26T21:57:32.227ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
122392025-08-26T21:57:32.227ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "1335e71e-0688-4ad6-b477-acf77354468e", "content-length": "682", "date": "Tue, 26 Aug 2025 21:57:31 GMT"} })
122402025-08-26T21:57:32.228ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
122412025-08-26T21:57:32.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
122422025-08-26T21:57:32.232ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
122432025-08-26T21:57:32.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
122442025-08-26T21:57:32.235ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
122452025-08-26T21:57:32.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
122462025-08-26T21:57:32.237ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
122472025-08-26T21:57:32.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
122482025-08-26T21:57:32.245ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 570.571061ms iteration = 24 status = {}
122492025-08-26T21:57:35.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 387 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 96842e6e-6f40-4195-bf6b-fbc1fcba66aa response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
122502025-08-26T21:57:36.665ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 140 reason = Timeout
122512025-08-26T21:57:36.665ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
122522025-08-26T21:57:36.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
122532025-08-26T21:57:36.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122542025-08-26T21:57:36.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
122552025-08-26T21:57:36.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
122562025-08-26T21:57:36.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
122572025-08-26T21:57:36.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
122582025-08-26T21:57:36.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.276777ms iteration = 140 status = {"chicken_switches_updated":false}
122592025-08-26T21:57:41.665ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 141 reason = Timeout
122602025-08-26T21:57:41.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
122612025-08-26T21:57:41.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
122622025-08-26T21:57:41.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122632025-08-26T21:57:41.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
122642025-08-26T21:57:41.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
122652025-08-26T21:57:41.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
122662025-08-26T21:57:41.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
122672025-08-26T21:57:41.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.785695ms iteration = 141 status = {"chicken_switches_updated":false}
122682025-08-26T21:57:41.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
122692025-08-26T21:57:41.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
122702025-08-26T21:57:41.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
122712025-08-26T21:57:41.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122722025-08-26T21:57:41.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
122732025-08-26T21:57:41.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
122742025-08-26T21:57:41.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
122752025-08-26T21:57:41.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
122762025-08-26T21:57:41.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
122772025-08-26T21:57:41.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122782025-08-26T21:57:41.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
122792025-08-26T21:57:41.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
122802025-08-26T21:57:41.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
122812025-08-26T21:57:41.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
122822025-08-26T21:57:41.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
122832025-08-26T21:57:41.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
122842025-08-26T21:57:41.688ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
122852025-08-26T21:57:41.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
122862025-08-26T21:57:41.704ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
122872025-08-26T21:57:41.704ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 30.363605ms iteration = 71 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
122882025-08-26T21:57:41.725ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 39.47244ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
122892025-08-26T21:57:45.351ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 354 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 11401343-ae9b-43fb-875a-7b3a0091f1e1 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
122902025-08-26T21:57:46.665ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 142 reason = Timeout
122912025-08-26T21:57:46.665ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
122922025-08-26T21:57:46.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
122932025-08-26T21:57:46.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122942025-08-26T21:57:46.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
122952025-08-26T21:57:46.673ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
122962025-08-26T21:57:46.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
122972025-08-26T21:57:46.676ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
122982025-08-26T21:57:46.676ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 10.342413ms iteration = 142 status = {"chicken_switches_updated":false}
122992025-08-26T21:57:51.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 143 reason = Timeout
123002025-08-26T21:57:51.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
123012025-08-26T21:57:51.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
123022025-08-26T21:57:51.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123032025-08-26T21:57:51.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
123042025-08-26T21:57:51.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
123052025-08-26T21:57:51.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
123062025-08-26T21:57:51.674ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
123072025-08-26T21:57:51.674ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
123082025-08-26T21:57:51.674ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.019901ms iteration = 143 status = {"chicken_switches_updated":false}
123092025-08-26T21:57:51.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
123102025-08-26T21:57:51.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
123112025-08-26T21:57:51.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123122025-08-26T21:57:51.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
123132025-08-26T21:57:51.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
123142025-08-26T21:57:51.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
123152025-08-26T21:57:51.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
123162025-08-26T21:57:51.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
123172025-08-26T21:57:51.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123182025-08-26T21:57:51.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
123192025-08-26T21:57:51.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
123202025-08-26T21:57:51.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
123212025-08-26T21:57:51.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
123222025-08-26T21:57:51.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
123232025-08-26T21:57:51.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
123242025-08-26T21:57:51.687ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
123252025-08-26T21:57:51.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
123262025-08-26T21:57:51.689ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.473138ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
123272025-08-26T21:57:51.697ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
123282025-08-26T21:57:51.697ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.268955ms iteration = 72 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
123292025-08-26T21:57:55.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 381 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 55222679-bf5c-4486-a529-65c65ca3f229 response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
123302025-08-26T21:57:56.665ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 144 reason = Timeout
123312025-08-26T21:57:56.665ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
123322025-08-26T21:57:56.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
123332025-08-26T21:57:56.667ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123342025-08-26T21:57:56.670ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
123352025-08-26T21:57:56.670ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
123362025-08-26T21:57:56.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
123372025-08-26T21:57:56.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
123382025-08-26T21:57:56.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.545411ms iteration = 144 status = {"chicken_switches_updated":false}
123392025-08-26T21:58:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
123402025-08-26T21:58:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
123412025-08-26T21:58:01.664ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
123422025-08-26T21:58:01.664ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
123432025-08-26T21:58:01.664ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
123442025-08-26T21:58:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
123452025-08-26T21:58:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
123462025-08-26T21:58:01.664ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 25 reason = Timeout
123472025-08-26T21:58:01.664ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
123482025-08-26T21:58:01.664ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
123492025-08-26T21:58:01.664ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
123502025-08-26T21:58:01.664ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
123512025-08-26T21:58:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
123522025-08-26T21:58:01.664ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-08-26 21:48:01.664564743 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
123532025-08-26T21:58:01.664ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
123542025-08-26T21:58:01.664ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
123552025-08-26T21:58:01.664ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
123562025-08-26T21:58:01.664ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
123572025-08-26T21:58:01.664ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
123582025-08-26T21:58:01.664ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
123592025-08-26T21:58:01.664ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
123602025-08-26T21:58:01.664ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:91 reason = blueprint disabled target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8
123612025-08-26T21:58:01.664ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 13.88\u{b5}s iteration = 14 status = {"enabled":false,"target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8"}
123622025-08-26T21:58:01.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
123632025-08-26T21:58:01.664ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("60abf731-31e1-4111-a3e2-65da89eaed87.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("84c849eb-0949-41d9-ad02-9a5509144564.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1585, tv_nsec: 524131578 } })
123642025-08-26T21:58:01.665ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("852fd1a3-de9a-47e8-a667-5e8dab880ad3.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("c68f1501-294c-4874-92ca-8fc0546a5870.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("d741a72f-8745-4723-9e80-e1e72987266b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:3::1)) }], valid_until: Instant { tv_sec: 1585, tv_nsec: 524224680 } })
123652025-08-26T21:58:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
123662025-08-26T21:58:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
123672025-08-26T21:58:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
123682025-08-26T21:58:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
123692025-08-26T21:58:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
123702025-08-26T21:58:01.665ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
123712025-08-26T21:58:01.665ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 994.693\u{b5}s iteration = 13 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
123722025-08-26T21:58:01.665ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 145 reason = Timeout
123732025-08-26T21:58:01.665ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
123742025-08-26T21:58:01.665ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } dns_group = internal
123752025-08-26T21:58:01.665ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.002883ms iteration = 13 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353"]}
123762025-08-26T21:58:01.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
123772025-08-26T21:58:01.665ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
123782025-08-26T21:58:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
123792025-08-26T21:58:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
123802025-08-26T21:58:01.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
123812025-08-26T21:58:01.667ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1585, tv_nsec: 526681556 } })
123822025-08-26T21:58:01.667ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-08-26 21:48:01.664564743 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
123832025-08-26T21:58:01.667ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1585, tv_nsec: 526847159 } })
123842025-08-26T21:58:01.667ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.10342ms iteration = 13 status = {"expiration":"2025-08-26T21:48:01.664564743Z","pruned":[]}
123852025-08-26T21:58:01.667ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1205
123862025-08-26T21:58:01.667ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123872025-08-26T21:58:01.667ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1205
123882025-08-26T21:58:01.668ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.373759ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
123892025-08-26T21:58:01.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
123902025-08-26T21:58:01.672ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
123912025-08-26T21:58:01.672ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
123922025-08-26T21:58:01.672ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
123932025-08-26T21:58:01.672ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
123942025-08-26T21:58:01.672ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 16 reason = Timeout
123952025-08-26T21:58:01.672ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
123962025-08-26T21:58:01.672ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 7.96\u{b5}s iteration = 16 status = "Disabled"
123972025-08-26T21:58:01.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
123982025-08-26T21:58:01.673ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
123992025-08-26T21:58:01.673ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 27 reason = Timeout
124002025-08-26T21:58:01.673ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
124012025-08-26T21:58:01.673ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Timeout
124022025-08-26T21:58:01.673ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
124032025-08-26T21:58:01.673ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
124042025-08-26T21:58:01.673ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
124052025-08-26T21:58:01.673ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
124062025-08-26T21:58:01.673ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
124072025-08-26T21:58:01.673ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
124082025-08-26T21:58:01.673ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 25 reason = Timeout
124092025-08-26T21:58:01.673ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
124102025-08-26T21:58:01.673ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
124112025-08-26T21:58:01.673ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
124122025-08-26T21:58:01.673ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
124132025-08-26T21:58:01.673ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
124142025-08-26T21:58:01.673ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
124152025-08-26T21:58:01.673ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
124162025-08-26T21:58:01.673ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
124172025-08-26T21:58:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
124182025-08-26T21:58:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
124192025-08-26T21:58:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
124202025-08-26T21:58:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
124212025-08-26T21:58:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
124222025-08-26T21:58:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
124232025-08-26T21:58:01.674ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
124242025-08-26T21:58:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
124252025-08-26T21:58:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
124262025-08-26T21:58:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
124272025-08-26T21:58:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
124282025-08-26T21:58:01.674ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
124292025-08-26T21:58:01.674ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124302025-08-26T21:58:01.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
124312025-08-26T21:58:01.674ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
124322025-08-26T21:58:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
124332025-08-26T21:58:01.675ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
124342025-08-26T21:58:01.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
124352025-08-26T21:58:01.676ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124362025-08-26T21:58:01.677ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124372025-08-26T21:58:01.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124382025-08-26T21:58:01.677ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124392025-08-26T21:58:01.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
124402025-08-26T21:58:01.680ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
124412025-08-26T21:58:01.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
124422025-08-26T21:58:01.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
124432025-08-26T21:58:01.681ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
124442025-08-26T21:58:01.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
124452025-08-26T21:58:01.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
124462025-08-26T21:58:01.682ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
124472025-08-26T21:58:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
124482025-08-26T21:58:01.683ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
124492025-08-26T21:58:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
124502025-08-26T21:58:01.683ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
124512025-08-26T21:58:01.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
124522025-08-26T21:58:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
124532025-08-26T21:58:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
124542025-08-26T21:58:01.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
124552025-08-26T21:58:01.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
124562025-08-26T21:58:01.685ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124572025-08-26T21:58:01.685ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
124582025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
124592025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
124602025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
124612025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
124622025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
124632025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
124642025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
124652025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
124662025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
124672025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
124682025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
124692025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
124702025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 13 reason = Timeout
124712025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
124722025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
124732025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 13 reason = Timeout
124742025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
124752025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
124762025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
124772025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
124782025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
124792025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
124802025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
124812025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
124822025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
124832025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 25 reason = Timeout
124842025-08-26T21:58:01.686ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
124852025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
124862025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 25 reason = Timeout
124872025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
124882025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
124892025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
124902025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 25 reason = Timeout
124912025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 11.231\u{b5}s iteration = 25 status = {"disabled":true,"errors":[],"sps":[]}
124922025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
124932025-08-26T21:58:01.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
124942025-08-26T21:58:01.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
124952025-08-26T21:58:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
124962025-08-26T21:58:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
124972025-08-26T21:58:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
124982025-08-26T21:58:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
124992025-08-26T21:58:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
125002025-08-26T21:58:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
125012025-08-26T21:58:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
125022025-08-26T21:58:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
125032025-08-26T21:58:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
125042025-08-26T21:58:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
125052025-08-26T21:58:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
125062025-08-26T21:58:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
125072025-08-26T21:58:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
125082025-08-26T21:58:01.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
125092025-08-26T21:58:01.692ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125102025-08-26T21:58:01.695ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 8.787478ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
125112025-08-26T21:58:01.697ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
125122025-08-26T21:58:01.697ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
125132025-08-26T21:58:01.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
125142025-08-26T21:58:01.697ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
125152025-08-26T21:58:01.697ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
125162025-08-26T21:58:01.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
125172025-08-26T21:58:01.697ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 10.62846ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
125182025-08-26T21:58:01.710ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
125192025-08-26T21:58:01.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
125202025-08-26T21:58:01.722ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
125212025-08-26T21:58:01.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
125222025-08-26T21:58:01.724ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125232025-08-26T21:58:01.728ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 3724495d-3cd2-4a88-87e4-560a40a8f5a8, lookup_type: ById(3724495d-3cd2-4a88-87e4-560a40a8f5a8) } result = Ok(())
125242025-08-26T21:58:01.728ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
125252025-08-26T21:58:01.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
125262025-08-26T21:58:01.729ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 42.341074ms iteration = 25 status = {"errors":[],"finish_invoked_ok":[]}
125272025-08-26T21:58:01.731ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125282025-08-26T21:58:01.734ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
125292025-08-26T21:58:01.734ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
125302025-08-26T21:58:01.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
125312025-08-26T21:58:01.739ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125322025-08-26T21:58:01.741ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125332025-08-26T21:58:01.741ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125342025-08-26T21:58:01.744ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
125352025-08-26T21:58:01.744ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 57.745032ms iteration = 13 status = {"by_rx":{},"error":null}
125362025-08-26T21:58:01.744ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
125372025-08-26T21:58:01.745ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
125382025-08-26T21:58:01.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
125392025-08-26T21:58:01.745ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125402025-08-26T21:58:01.746ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
125412025-08-26T21:58:01.746ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
125422025-08-26T21:58:01.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
125432025-08-26T21:58:01.747ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
125442025-08-26T21:58:01.748ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 62.13385ms iteration = 25 status = {"errors":[],"requests_created_ok":[]}
125452025-08-26T21:58:01.749ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
125462025-08-26T21:58:01.749ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
125472025-08-26T21:58:01.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
125482025-08-26T21:58:01.757ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125492025-08-26T21:58:01.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
125502025-08-26T21:58:01.761ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
125512025-08-26T21:58:01.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
125522025-08-26T21:58:01.763ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
125532025-08-26T21:58:01.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
125542025-08-26T21:58:01.765ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125552025-08-26T21:58:01.766ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125562025-08-26T21:58:01.769ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
125572025-08-26T21:58:01.769ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
125582025-08-26T21:58:01.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
125592025-08-26T21:58:01.769ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125602025-08-26T21:58:01.769ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
125612025-08-26T21:58:01.769ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
125622025-08-26T21:58:01.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
125632025-08-26T21:58:01.772ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
125642025-08-26T21:58:01.772ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
125652025-08-26T21:58:01.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
125662025-08-26T21:58:01.772ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125672025-08-26T21:58:01.774ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125682025-08-26T21:58:01.775ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
125692025-08-26T21:58:01.775ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
125702025-08-26T21:58:01.775ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125712025-08-26T21:58:01.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
125722025-08-26T21:58:01.775ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125732025-08-26T21:58:01.779ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
125742025-08-26T21:58:01.779ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
125752025-08-26T21:58:01.779ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
125762025-08-26T21:58:01.779ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
125772025-08-26T21:58:01.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
125782025-08-26T21:58:01.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
125792025-08-26T21:58:01.779ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
125802025-08-26T21:58:01.779ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
125812025-08-26T21:58:01.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
125822025-08-26T21:58:01.780ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
125832025-08-26T21:58:01.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
125842025-08-26T21:58:01.781ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
125852025-08-26T21:58:01.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = alert_dispatcher resource = Database result = Ok(())
125862025-08-26T21:58:01.783ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
125872025-08-26T21:58:01.783ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 118.190414ms iteration = 145 status = {"chicken_switches_updated":false}
125882025-08-26T21:58:01.784ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 110.765917ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
125892025-08-26T21:58:01.788ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
125902025-08-26T21:58:01.788ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 0 version = 2
125912025-08-26T21:58:01.788ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
125922025-08-26T21:58:01.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
125932025-08-26T21:58:01.794ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
125942025-08-26T21:58:01.794ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
125952025-08-26T21:58:01.794ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 120.806103ms iteration = 6 status = {"nsuccess":5}
125962025-08-26T21:58:01.794ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b24d99de-84d1-465b-b2c7-d14bbca4bf26", "content-length": "26", "date": "Tue, 26 Aug 2025 21:58:01 GMT"} })
125972025-08-26T21:58:01.794ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
125982025-08-26T21:58:01.796ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
125992025-08-26T21:58:01.796ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
126002025-08-26T21:58:01.796ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 109.453487ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
126012025-08-26T21:58:01.796ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
126022025-08-26T21:58:01.797ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 110.926471ms iteration = 13 status = {"dispatched":[],"errors":[],"glob_version":"183.0.0","globs_reprocessed":{},"no_receivers":[]}
126032025-08-26T21:58:01.797ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-dd0c655a-95c2-48fc-b4c3-9733ed4b0faa.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1585, tv_nsec: 657114836 } })
126042025-08-26T21:58:01.799ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1205 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
126052025-08-26T21:58:01.799ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
126062025-08-26T21:58:01.799ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
126072025-08-26T21:58:01.799ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
126082025-08-26T21:58:01.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
126092025-08-26T21:58:01.800ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126102025-08-26T21:58:01.803ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
126112025-08-26T21:58:01.803ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
126122025-08-26T21:58:01.803ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
126132025-08-26T21:58:01.803ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
126142025-08-26T21:58:01.803ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
126152025-08-26T21:58:01.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
126162025-08-26T21:58:01.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
126172025-08-26T21:58:01.807ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = dfb404ab-acb8-43de-afe8-b429af0c161d dns_zone_name = oxide.test found_so_far = 4 version = 2
126182025-08-26T21:58:01.807ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
126192025-08-26T21:58:01.807ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-08-26 21:46:05.296205 UTC dns_group = external generation = 2
126202025-08-26T21:58:01.807ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 142.672286ms iteration = 13 status = {"generation":2}
126212025-08-26T21:58:01.807ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
126222025-08-26T21:58:01.807ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 original_time_created = 2025-08-26 21:45:55.109894 UTC target_id = 3724495d-3cd2-4a88-87e4-560a40a8f5a8 time_created = 2025-08-26 21:45:55.109894 UTC
126232025-08-26T21:58:01.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
126242025-08-26T21:58:01.807ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 133.914529ms iteration = 73 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"3724495d-3cd2-4a88-87e4-560a40a8f5a8","time_created":"2025-08-26T21:45:55.109894Z"}
126252025-08-26T21:58:01.807ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
126262025-08-26T21:58:01.807ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
126272025-08-26T21:58:01.807ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 121.180541ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
126282025-08-26T21:58:01.810ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
126292025-08-26T21:58:01.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
126302025-08-26T21:58:01.812ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126312025-08-26T21:58:01.815ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: dd0c655a-95c2-48fc-b4c3-9733ed4b0faa, lookup_type: ById(dd0c655a-95c2-48fc-b4c3-9733ed4b0faa) } result = Ok(())
126322025-08-26T21:58:01.815ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
126332025-08-26T21:58:01.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
126342025-08-26T21:58:01.818ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 144.971818ms iteration = 25 status = 0
126352025-08-26T21:58:01.823ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
126362025-08-26T21:58:01.824ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
126372025-08-26T21:58:01.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
126382025-08-26T21:58:01.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
126392025-08-26T21:58:01.826ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
126402025-08-26T21:58:01.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
126412025-08-26T21:58:01.835ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
126422025-08-26T21:58:01.835ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 161.680885ms iteration = 25 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
126432025-08-26T21:58:01.838ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
126442025-08-26T21:58:01.838ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
126452025-08-26T21:58:01.838ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
126462025-08-26T21:58:01.838ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
126472025-08-26T21:58:01.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
126482025-08-26T21:58:01.840ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126492025-08-26T21:58:01.846ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
126502025-08-26T21:58:01.846ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
126512025-08-26T21:58:01.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
126522025-08-26T21:58:01.850ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126532025-08-26T21:58:01.852ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
126542025-08-26T21:58:01.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
126552025-08-26T21:58:01.854ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
126562025-08-26T21:58:01.854ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
126572025-08-26T21:58:01.854ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
126582025-08-26T21:58:01.854ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
126592025-08-26T21:58:01.854ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 180.438587ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
126602025-08-26T21:58:01.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
126612025-08-26T21:58:01.854ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126622025-08-26T21:58:01.857ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 170.751909ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
126632025-08-26T21:58:01.857ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
126642025-08-26T21:58:01.857ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
126652025-08-26T21:58:01.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
126662025-08-26T21:58:01.863ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126672025-08-26T21:58:01.866ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
126682025-08-26T21:58:01.866ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
126692025-08-26T21:58:01.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
126702025-08-26T21:58:01.880ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
126712025-08-26T21:58:01.880ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 193.534643ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
126722025-08-26T21:58:01.884ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 197.606884ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
126732025-08-26T21:58:01.889ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 202.73929ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
126742025-08-26T21:58:01.905ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
126752025-08-26T21:58:01.905ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
126762025-08-26T21:58:01.905ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
126772025-08-26T21:58:01.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
126782025-08-26T21:58:01.907ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126792025-08-26T21:58:01.908ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
126802025-08-26T21:58:01.908ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
126812025-08-26T21:58:01.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
126822025-08-26T21:58:01.911ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
126832025-08-26T21:58:01.911ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
126842025-08-26T21:58:01.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
126852025-08-26T21:58:01.912ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
126862025-08-26T21:58:01.912ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 226.319492ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
126872025-08-26T21:58:01.914ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = 8e720f02-dde8-4423-a6e3-91698373805a dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
126882025-08-26T21:58:01.914ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
126892025-08-26T21:58:01.914ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-08-26 21:46:04.065217 UTC dns_group = internal generation = 1
126902025-08-26T21:58:01.914ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 249.950485ms iteration = 13 status = {"generation":1}
126912025-08-26T21:58:01.924ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
126922025-08-26T21:58:01.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
126932025-08-26T21:58:01.926ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126942025-08-26T21:58:01.929ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
126952025-08-26T21:58:01.929ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
126962025-08-26T21:58:01.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
126972025-08-26T21:58:01.937ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 263.48203ms iteration = 27 status = {"physical_disks_added":0}
126982025-08-26T21:58:01.939ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-08-26T21:45:59.866217Z, time_modified: 2025-08-26T21:45:59.866217Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, db_silo: Silo { identity: SiloIdentity { id: 9396aa55-de6f-4d6f-96db-d28e1c76fa9c, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-08-26T21:46:05.245521Z, time_modified: 2025-08-26T21:46:05.245521Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)) }, tls_certs: [TlsCertificate { digest: "e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af" }] }) }
126992025-08-26T21:58:01.939ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 274.871037ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]}},"default_endpoint":{"silo_id":"9396aa55-de6f-4d6f-96db-d28e1c76fa9c","tls_certs":["e8529d720e4b0de099753dcdb373abb870af04baacd7c7dd850f5254858081af"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
127002025-08-26T21:58:01.953ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1213 zone_address = fd00:1122:3344:101::2
127012025-08-26T21:58:01.953ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
127022025-08-26T21:58:01.953ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "03cfc0d4-48b0-4a17-9ee6-f764d107f9f8", "content-length": "26", "date": "Tue, 26 Aug 2025 21:58:01 GMT"} })
127032025-08-26T21:58:01.953ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1216 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
127042025-08-26T21:58:01.954ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
127052025-08-26T21:58:01.955ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
127062025-08-26T21:58:01.984ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
127072025-08-26T21:58:01.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
127082025-08-26T21:58:01.987ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 322.769416ms iteration = 25 status = null
127092025-08-26T21:58:02.022ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
127102025-08-26T21:58:02.022ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
127112025-08-26T21:58:02.022ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "c81acc16-50db-4437-aeab-5e3f22ecc608", "content-length": "372", "date": "Tue, 26 Aug 2025 21:58:01 GMT"} })
127122025-08-26T21:58:02.022ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
127132025-08-26T21:58:02.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
127142025-08-26T21:58:02.036ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
127152025-08-26T21:58:02.036ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
127162025-08-26T21:58:02.036ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "3b4b1fdd-4490-4e3a-8067-3f43e467ece3", "date": "Tue, 26 Aug 2025 21:58:01 GMT"} })
127172025-08-26T21:58:02.036ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
127182025-08-26T21:58:02.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
127192025-08-26T21:58:02.039ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
127202025-08-26T21:58:02.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
127212025-08-26T21:58:02.042ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
127222025-08-26T21:58:02.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
127232025-08-26T21:58:02.044ZINFOnexus (BackgroundTasks): VPC route manager sled dd0c655a-95c2-48fc-b4c3-9733ed4b0faa push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
127242025-08-26T21:58:02.044ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
127252025-08-26T21:58:02.044ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "3a7946a3-ba03-438b-9980-a8fc844f698e", "date": "Tue, 26 Aug 2025 21:58:01 GMT"} })
127262025-08-26T21:58:02.046ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 359.74508ms iteration = 25 status = {}
127272025-08-26T21:58:02.080ZDEBGnexus (BackgroundTasks): client request SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
127282025-08-26T21:58:02.080ZDEBGnexus (BackgroundTasks): client response SledAgent = dd0c655a-95c2-48fc-b4c3-9733ed4b0faa background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "561a3949-f41e-4fbe-8aa1-f8f9048d140b", "content-length": "691", "date": "Tue, 26 Aug 2025 21:58:01 GMT"} })
127292025-08-26T21:58:02.080ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = sock
127302025-08-26T21:58:02.080ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = sock
127312025-08-26T21:58:02.080ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = sock
127322025-08-26T21:58:02.082ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 409.134922ms iteration = 25 status = {}
127332025-08-26T21:58:02.109ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1213 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 zone_address = fd00:1122:3344:101::2
127342025-08-26T21:58:02.109ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
127352025-08-26T21:58:02.109ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "90069e7d-c542-430d-af34-f7a57a33704d", "content-length": "26", "date": "Tue, 26 Aug 2025 21:58:01 GMT"} })
127362025-08-26T21:58:02.109ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1216 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
127372025-08-26T21:58:02.111ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1250 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
127382025-08-26T21:58:02.122ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 458.430134ms iteration = 25 status = {}
127392025-08-26T21:58:02.134ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
127402025-08-26T21:58:02.140ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
127412025-08-26T21:58:02.140ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "d836624c-1ab2-4a1f-96a1-f00b22e29cd2", "date": "Tue, 26 Aug 2025 21:58:01 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
127422025-08-26T21:58:02.140ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
127432025-08-26T21:58:02.142ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "697b966c-97e8-44e4-9644-ef59184c0666", "date": "Tue, 26 Aug 2025 21:58:01 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
127442025-08-26T21:58:02.142ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
127452025-08-26T21:58:02.144ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
127462025-08-26T21:58:02.145ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 480.531412ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
127472025-08-26T21:58:02.254ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
127482025-08-26T21:58:02.255ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "7bd6066c-e3d8-412b-8296-79c92e8e3827", "date": "Tue, 26 Aug 2025 21:58:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
127492025-08-26T21:58:02.255ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
127502025-08-26T21:58:02.256ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "101a3bcb-d142-4040-8087-84cf965c8428", "date": "Tue, 26 Aug 2025 21:58:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
127512025-08-26T21:58:02.256ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
127522025-08-26T21:58:02.258ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "615eb809-d709-4476-a0f0-147b108054d9", "date": "Tue, 26 Aug 2025 21:58:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
127532025-08-26T21:58:02.258ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
127542025-08-26T21:58:02.260ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353] }
127552025-08-26T21:58:02.260ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 595.914543ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
127562025-08-26T21:58:02.313ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
127572025-08-26T21:58:02.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
127582025-08-26T21:58:02.315ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: d24c5c55-3a21-4a91-aeb4-45362e822df6, rack_id: e37a46f2-f789-4ca4-9841-f8c6dd705964, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(88d84f37-a4c8-4abc-af1f-27987b0d39d9) } rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
127592025-08-26T21:58:02.315ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
127602025-08-26T21:58:02.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
127612025-08-26T21:58:02.328ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
127622025-08-26T21:58:02.328ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 88d84f37-a4c8-4abc-af1f-27987b0d39d9
127632025-08-26T21:58:02.329ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
127642025-08-26T21:58:02.329ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
127652025-08-26T21:58:02.329ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
127662025-08-26T21:58:02.329ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
127672025-08-26T21:58:02.329ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
127682025-08-26T21:58:02.329ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
127692025-08-26T21:58:02.329ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
127702025-08-26T21:58:02.329ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
127712025-08-26T21:58:02.329ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964
127722025-08-26T21:58:02.329ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
127732025-08-26T21:58:02.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
127742025-08-26T21:58:02.331ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch_location = Switch0
127752025-08-26T21:58:02.331ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
127762025-08-26T21:58:02.438ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "9c981c47-4922-45f3-90d9-6fe701f2f630", "date": "Tue, 26 Aug 2025 21:58:01 GMT"} })
127772025-08-26T21:58:02.438ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 switch = Switch0
127782025-08-26T21:58:02.438ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
127792025-08-26T21:58:02.438ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = e37a46f2-f789-4ca4-9841-f8c6dd705964 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ef3bd97-2c1c-40aa-b6de-ec9957c2d762", "content-length": "682", "date": "Tue, 26 Aug 2025 21:58:01 GMT"} })
127802025-08-26T21:58:02.438ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
127812025-08-26T21:58:02.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
127822025-08-26T21:58:02.443ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
127832025-08-26T21:58:02.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
127842025-08-26T21:58:02.446ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
127852025-08-26T21:58:02.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
127862025-08-26T21:58:02.449ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
127872025-08-26T21:58:02.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
127882025-08-26T21:58:02.457ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 783.764628ms iteration = 25 status = {}
127892025-08-26T21:58:05.352ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 352 local_addr = [fd00:1122:3344:101::a]:35154 method = GET remote_addr = [fd00:1122:3344:101::d]:59465 req_id = 33efb2ed-38dd-4bd9-a111-790fbda2f4aa response_code = 200 uri = /4996dcc4-c530-4f64-b407-fa372462bd9e
127902025-08-26T21:58:06.666ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 146 reason = Timeout
127912025-08-26T21:58:06.666ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
127922025-08-26T21:58:06.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
127932025-08-26T21:58:06.668ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127942025-08-26T21:58:06.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
127952025-08-26T21:58:06.671ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
127962025-08-26T21:58:06.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
127972025-08-26T21:58:06.673ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
127982025-08-26T21:58:06.673ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.764462ms iteration = 146 status = {"chicken_switches_updated":false}